Klassifikation von Corona Patienten

Einleitung

Im Rahmen des Algorithmus und Statistik 2 Lab wird hier ein Datensatz zur vertieften Analyse und Modellierung vorgestellt. Die Gruppe (Gruppe L) setzt sich aus folgenden Mitgliedern zusammen:

• Bernd Kaufmann (1910837033)
• Tobias Gosch (1910837735)
• Reto Heller (1910837262)
• Louis Heublein (1910837388)
• Julian Bialas (1910837917)

Datensatz

Beschreibung

Die Datengrundlage für das Projekt kann in einer Kaggle Challenge gefunden werden. Die Daten beinhalten anonymisierte Informationen über Verdachtsfälle von Corona aus dem Israelita Albert Einstein Hospital in Sauó Paulo, Brasilien. Hier wurden 5644 Patienten auf Corona getestet. Auch werden Daten über Vorerkrankungen, Blutwerten, Alter und Weiteres zur Verfügung gestellt. Der Datensatz wurde bereits im Kontext des Machine Learning Lab 1 aufbereitet; die dort durchgeführten Schritte sind unten zu finden. Die Spalten, welche übergeblieben sind sind folgende:

• target: corona test resultat
• Patient.age.quantile: Alter des Patienten als quantil
• sickness: Patient hat Vorerkrankung
• Patient.addmited.to.regular.ward..1.yes..0.no.: Patient wurde im Krankenhaus aufgenommen
• Patient.addmited.to.semi.intensive.unit..1.yes..0.no.: Patent wurde auf eine Vorstufe der Intensivstation aufgenommen
• Patient.addmited.to.intensive.care.unit..1.yes..0.no.: Patient wurde auf Intensivstation aufgenommen
• Hematocrit: Hämatokrit Konzentration
• Platelets: Thrombozyten Konzentration
• Mean.platelet.volume: Mittlere Thrombozyten Volumen
• Lymphocytes: Lymphocyten Konzentration
• Mean.corpuscular.hemoglobin.concentrationÂ..MCHC.: a measure of the concentration of haemoglobin in a given volume of packed red blood cell.
• Leukocytes: Leukocytes Konzentration
• Basophils: white blood cells from the bone marrow, Konzentration
• Mean.corpuscular.hemoglobin..MCH.
• Eosinophils
• Monocytes: Monozyten Konzentration
• Red.blood.cell.distribution.width..RDW.: Spannweite der Verteilung der roten Blutkörper

Alle kontinuierlichen Daten sind zentriert. Die Daten beinhalten nur 8,4% corona-positive Patienten.

Cleaning

In einem jupyter notebook wurden schon einige Schritte des Preprocessing durchgeführt. Die meisten davon waren notwendig, um die große Anzahl an Nullwerten zu entfernen/ersetzen. Folgende Schritte wurden durchgeführt:

• Alle Spalten mit mehr als 90% NAs wurden entfernt
• Die Patienten ID wurde entfernt
• Konstante Spalten wurden entfernt
• Spalten die zu einer anderen zu 0.85 Korrelation zeigen wurden entfernt
• Zeilen die einen hohen Wert an NAs beinhalten wurden entfernt
• Die Spalte sickness wurde erstellt. Sie beträgt 1, falls einer von den vielen durchgeführten (nicht-corona) Tests positiv ist. Sprich, wenn eine Vorerkrankung besteht.
• Die restlichen NAs wurden mit KNN imputiert. Da viele Werte nur NAs als Nachbarn haben, wurde meist der mean als Alternative genommen. Das resultiert darin, dass extrem viele Werte um 0 (Daten sind zentriert) liegen.

Vorbereitung in R

Daten einlesen

library(kableExtra)
library(knitr)
library(dplyr)
library(caret)

data_clean <- read.csv("data/clean/data_clean.csv")
str(data_clean)

## 'data.frame':    532 obs. of  17 variables:
##  $ Patient.age.quantile                                 : int  17 1 9 11 0 13 14 9 8 17 ...
##  $ target                                               : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ Patient.addmited.to.regular.ward..1.yes..0.no.       : int  0 0 0 0 0 0 0 1 0 0 ...
##  $ Patient.addmited.to.semi.intensive.unit..1.yes..0.no.: int  0 1 0 0 0 0 0 0 0 0 ...
##  $ Patient.addmited.to.intensive.care.unit..1.yes..0.no.: int  0 0 0 0 0 0 0 0 0 0 ...
##  $ sickness                                             : int  1 0 1 1 1 1 0 1 0 1 ...
##  $ Hematocrit                                           : num  0.2365 -1.5717 -0.7477 0.9918 -0.0742 ...
##  $ Platelets                                            : num  -0.5174 1.4297 -0.4295 0.073 -0.0326 ...
##  $ Mean.platelet.volume                                 : num  0.01068 -1.67222 -0.21371 -0.55029 -0.00447 ...
##  $ Lymphocytes                                          : num  0.31837 -0.00574 -1.11451 0.04544 -0.07253 ...
##  $ Mean.corpuscular.hemoglobin.concentration..MCHC.     : num  -0.9508 3.3311 0.5429 -0.4529 -0.0459 ...
##  $ Leukocytes                                           : num  -0.0946 0.3646 -0.8849 -0.2115 0.0432 ...
##  $ Basophils                                            : num  -0.2238 -0.2238 0.0817 -0.8347 -0.0251 ...
##  $ Mean.corpuscular.hemoglobin..MCH.                    : num  -0.2923 0.1782 1.7463 0.335 -0.0515 ...
##  $ Eosinophils                                          : num  1.4822 1.0186 -0.667 -0.7091 0.0103 ...
##  $ Monocytes                                            : num  0.3575 0.0687 1.2768 -0.2202 0.0421 ...
##  $ Red.blood.cell.distribution.width..RDW.              : num  -0.625 -0.979 -1.067 0.171 0.086 ...

data_clean %>% head() %>% kable() %>% kable_styling(font_size = 6)

Patient.age.quantile	target	Patient.addmited.to.regular.ward..1.yes..0.no.	Patient.addmited.to.semi.intensive.unit..1.yes..0.no.	Patient.addmited.to.intensive.care.unit..1.yes..0.no.	sickness	Hematocrit	Platelets	Mean.platelet.volume	Lymphocytes	Mean.corpuscular.hemoglobin.concentration..MCHC.	Leukocytes	Basophils	Mean.corpuscular.hemoglobin..MCH.	Eosinophils	Monocytes	Red.blood.cell.distribution.width..RDW.
17	0	0	0	0	1	0.2365154	-0.5174130	0.0106766	0.3183658	-0.9507903	-0.0946103	-0.2237665	-0.2922693	1.4821582	0.3575467	-0.6250727
1	0	0	1	0	0	-1.5716822	1.4296675	-1.6722218	-0.0057380	3.3310707	0.3645505	-0.2237665	0.1781750	1.0186250	0.0686515	-0.9788991
9	0	0	0	0	1	-0.7476931	-0.4294803	-0.2137107	-1.1145138	0.5428824	-0.8849232	0.0816925	1.7463233	-0.6669502	1.2767589	-1.0673550
11	0	0	0	0	1	0.9918382	0.0729920	-0.5502895	0.0454363	-0.4528995	-0.2114877	-0.8346847	0.3349894	-0.7090895	-0.2202439	0.1710353
0	0	0	0	0	1	-0.0742315	-0.0325821	-0.0044684	-0.0725255	-0.0458798	0.0432139	-0.0251347	-0.0514773	0.0102734	0.0421004	0.0859626
13	0	0	0	0	1	1.0147264	-0.1782442	0.7960289	-0.7307069	-0.3533190	-0.0751308	2.5253651	0.5440767	0.2179768	0.0686515	0.1710353

transform_type <- function(df){
  # iterate over columns 
  for (col_oi in colnames(df)){
    
    # and transform to factor (if it has little unique values)
    if (df[, col_oi] %>% unique() %>% length() < 10){
      df[, col_oi] <- as.factor(df[, col_oi])
      
    }else{
      # else transform to numeric
      df[, col_oi] <- as.numeric(df[, col_oi])
    }
  }
  return(df)
}

data_clean <- transform_type(data_clean)
print(str(data_clean))

## 'data.frame':    532 obs. of  17 variables:
##  $ Patient.age.quantile                                 : num  17 1 9 11 0 13 14 9 8 17 ...
##  $ target                                               : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ Patient.addmited.to.regular.ward..1.yes..0.no.       : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 2 1 1 ...
##  $ Patient.addmited.to.semi.intensive.unit..1.yes..0.no.: Factor w/ 2 levels "0","1": 1 2 1 1 1 1 1 1 1 1 ...
##  $ Patient.addmited.to.intensive.care.unit..1.yes..0.no.: Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ sickness                                             : Factor w/ 2 levels "0","1": 2 1 2 2 2 2 1 2 1 2 ...
##  $ Hematocrit                                           : num  0.2365 -1.5717 -0.7477 0.9918 -0.0742 ...
##  $ Platelets                                            : num  -0.5174 1.4297 -0.4295 0.073 -0.0326 ...
##  $ Mean.platelet.volume                                 : num  0.01068 -1.67222 -0.21371 -0.55029 -0.00447 ...
##  $ Lymphocytes                                          : num  0.31837 -0.00574 -1.11451 0.04544 -0.07253 ...
##  $ Mean.corpuscular.hemoglobin.concentration..MCHC.     : num  -0.9508 3.3311 0.5429 -0.4529 -0.0459 ...
##  $ Leukocytes                                           : num  -0.0946 0.3646 -0.8849 -0.2115 0.0432 ...
##  $ Basophils                                            : num  -0.2238 -0.2238 0.0817 -0.8347 -0.0251 ...
##  $ Mean.corpuscular.hemoglobin..MCH.                    : num  -0.2923 0.1782 1.7463 0.335 -0.0515 ...
##  $ Eosinophils                                          : num  1.4822 1.0186 -0.667 -0.7091 0.0103 ...
##  $ Monocytes                                            : num  0.3575 0.0687 1.2768 -0.2202 0.0421 ...
##  $ Red.blood.cell.distribution.width..RDW.              : num  -0.625 -0.979 -1.067 0.171 0.086 ...
## NULL

Visualization

Hier erkennt man recht gut, dass bei allen Blutwerten so wenig Daten vorhanden waren, dass es beim KNN-imputieren mit dem Mittelwert (ca. 0) ersetzt wurde.

plot_col <- function(df, col){
  g <- ggplot(data = df, mapping = aes_string(col)) 
  if(is.numeric(df[,col])){
    g <- g + geom_histogram(position = "identity", aes(fill=target))
  }else{
    g <- g + geom_histogram(stat = "count", aes(fill=target))
  }
  print(g)
}

for (col in colnames(data_clean)){
  plot_col(data_clean, col)
}

Train Test Split

set.seed(3456)
train_idx <- createDataPartition(data_clean$target, p = .8, 
                                  list = FALSE, 
                                  times = 1)

data_train <- data_clean[train_idx, ]
data_test <- data_clean[-train_idx, ]

print(dim(data_train))

## [1] 427  17

print(dim(data_test))

## [1] 105  17

# write.csv(x = data_train, file = "data/clean/train.csv", row.names = F)
# write.csv(x = data_test, file = "data/clean/test.csv", row.names = F)

Modellierbarkeit

first_model <-  glm(target ~ sickness + Patient.age.quantile + Hematocrit, data = data_train, family = "binomial")
print(summary(first_model))

## 
## Call:
## glm(formula = target ~ sickness + Patient.age.quantile + Hematocrit, 
##     family = "binomial", data = data_train)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.1433  -0.5064  -0.1623  -0.0767   3.5838  
## 
## Coefficients:
##                      Estimate Std. Error z value Pr(>|z|)    
## (Intercept)          -2.64987    0.48780  -5.432 5.56e-08 ***
## sickness1            -3.37686    0.73459  -4.597 4.29e-06 ***
## Patient.age.quantile  0.11574    0.03481   3.325 0.000886 ***
## Hematocrit            0.43911    0.18603   2.360 0.018252 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 296.05  on 426  degrees of freedom
## Residual deviance: 216.87  on 423  degrees of freedom
## AIC: 224.87
## 
## Number of Fisher Scoring iterations: 7

preds <- predict(object = first_model, newdata = data_test, type = "response")
print(preds[1:10])

##           5           7           9          17          18          32 
## 0.002330532 0.330809774 0.165192929 0.002615753 0.009281317 0.009281317 
##          41          59          64          69 
## 0.005224621 0.206027145 0.021674737 0.009150683

Ziele und Erwartungen

Das Ziel dieses Projektes ist es, ein Modell zu entwerfen, dass basierend auf den obig genannten Spalten den Ausgang des Corona Tests vorhersagen kann. Das bisherige Modell hat zwar eine Präzision von 0.91, jedoch einen recall von nur 0.33. Diesen gilt es zu heben. Die große Anzahl an mean imputed Werten (siehe Beschreibung oben) könnten ein Verhängnis werden. Wir haben uns zum Ziel gesetzt ein Modell zu entwickeln, dass zuverlässig unterscheiden kann zwischen Corona Infizierten und Nicht Corona Infizierten. Wir sind uns bewusst, dass dies ein sehr schwieriges Unterfangen ist. Eventuell gelingt es uns ein Modell zu entwickeln, dass für bereits eine grobe Klassifikation vornehmen kann. Damit im Nachgang für die “unsicheren” Patienten nach einem zweiten Test Gewissheit herrscht.

Feature engineering

Im unten abgebildeten Code beschäftigen wir uns mit feature engineering. Hierbei wählen wir zunächst die interressanten numerischen features aus und transformieren und kombinieren sie so, dass die corona erkrankten im Schnitt einen höhrer Wert erlangen. So generieren wir einige features. Bevor wir foreward selection anwenden, um zwei features hinzuzufügen, entfernen wir zwei durch backward selection.

###########################
# get data
###########################

data_train <- read.csv("data/clean/train.csv")
data_test <- read.csv("data/clean/test.csv")

transform_type <- function(df){
  # iterate over columns 
  for (col_oi in colnames(df)){
    
    # and transform to factor (if it has little unique values)
    if (df[, col_oi] %>% unique() %>% length() < 10){
      df[, col_oi] <- as.factor(df[, col_oi])
      
    }else{
      # else transform to numeric
      df[, col_oi] <- as.numeric(df[, col_oi])
    }
  }
  return(df)
}

#####
# upsample
#####

data_train <- transform_type(data_train)
data_test <- transform_type(data_test)

data_train_up <- upSample(x = data_train[, -ncol(data_train)],
                         y = data_train$target)
data_train_up <- data_train_up %>%
  select(-Class)



#############
# interesting features
#############

# look at the interesting features and first map them to the intervall [0,1]. Then (we want the final)
# variable have large values for corona patients) map small values to large ones (1-x). Finally 
# take the e function to guarantee that all values are positive (the upper described transformation could
# otherwise result in negative values for the test data)

extract_feat <- function(df){
  feat_oi <- list(
    "age" = exp((df$Patient.age.quantile - min(data_train_up$Patient.age.quantile))/(max(data_train_up$Patient.age.quantile) - min(data_train_up$Patient.age.quantile))),
    "plat" = exp(1 - (df$Platelets - min(data_train_up$Platelets))/(max(data_train_up$Platelets) - min(data_train_up$Platelets))),
    "leuk" = exp(1 - (df$Leukocytes - min(data_train_up$Leukocytes))/(max(data_train_up$Leukocytes) - min(data_train_up$Leukocytes))),
    "eos" = exp(1 - (df$Eosinophils - min(data_train_up$Eosinophils))/(max(data_train_up$Eosinophils) - min(data_train_up$Eosinophils)))
  )
  return(feat_oi)
}

feat_train <- extract_feat(data_train_up)
feat_test <- extract_feat(data_test)

# create all possible combinations of the three variables
comb_table <- combn(x = names(feat_train), 3)


# function that multiplies the features and returns vector of the result
create_feature <- function(name_vec, feat_oi){
  res <- 1
  for(name in name_vec){
    res <- res * feat_oi[[name]]
  }
  return(res)
}


# function that returns a list with all the combinations. Each element in list has a proper name.
# e.g (aes_dfs_dsf for the vector aes dfs dsf)
create_namevecs <- function(){
  name_vec_list <- list()
  list_names <- c()
  for (i in 1:ncol(comb_table)){
    list_names <- c(list_names, paste(comb_table[,i], collapse = "_"))
    name_vec_list[[length(name_vec_list) + 1]] <- comb_table[,i]
  }
  # add combination of all four cols 
  list_names <- c(list_names, paste(names(feat_train), collapse = "_"))
  name_vec_list[[length(name_vec_list) + 1]] <- names(feat_train)

  names(name_vec_list) <- list_names
  return(name_vec_list)
}

feat_list <- create_namevecs()

#####
# create the feature data frame for the training and test
#####
for (feat_name in names(feat_list)){
  eval(parse(text=paste0(feat_name, "= create_feature(feat_list[[feat_name]], feat_train)")))
}

eval(parse(text=paste0("feat_df_train = data.frame(", paste(names(feat_list), collapse = ','),")")))

for (feat_name in names(feat_list)){
  eval(parse(text=paste0(feat_name, "= create_feature(feat_list[[feat_name]], feat_test)")))
}

eval(parse(text=paste0("feat_df_test = data.frame(", paste(names(feat_list), collapse = ','),")")))

####################
# plot created features
####################

for (col in colnames(feat_df_train)){
  plot_df <- data.frame(target = data_train_up$target, feat = feat_df_train[, col])
  print(ggplot(data = plot_df) + geom_histogram(mapping = aes_string(fill = "target", x = "feat")) + xlab(col))
}

for (col in colnames(feat_df_train)){
  plot_df <- data.frame(target = data_test$target, feat = feat_df_test[, col])
  print(ggplot(data = plot_df) + geom_histogram(mapping = aes_string(fill = "target", x = "feat")) + xlab(col))
}


####################
# feature selection
####################

# to evaluate the the dataframe we will create a simple svm with similar parameters to @Louis svm. 
# returns the test accuracy. Ideally we would do this on an validation dataframe, but the dataset is 
# too small for that. 
eval_df <- function(data_train_oi, data_test_oi){
  set.seed(123)
  
  fitControl <- trainControl(## 10-fold CV
    method = "repeatedcv",
    number = 10,
    ## repeated ten times
    repeats = 3)
  
  svm_fit_radial <- train(target ~ ., data = data_train_oi, 
                          method = "svmRadial", 
                          trainControl = fitControl)
  
  prediction_radial <- svm_fit_radial %>% predict(data_test_oi)
  return(mean(prediction_radial == data_test_oi$target))
}


###################
# Backward elimination
###################

# initial values
final_df_train <- data_train_up
final_df_test <- data_test
unselected_feat <- colnames(feat_df_train)
acc_thresh <- eval_df(data_train_oi = data_train_up, 
                      data_test_oi = data_test)
selected_feat <- c()
dropped_feat <- c()

# here we will drop a feature in each interation (in case it makes an improvement to the accuracy)
for (iter in c(1,2)) {
  feat_to_drop <- NA
  for (feat_oi in colnames(final_df_train)[colnames(final_df_train) != "target"]) {
    tmp_df_train <- final_df_train[,! colnames(final_df_train) %in% c(feat_oi)]
    tmp_df_test <- final_df_test[,! colnames(final_df_test) %in% c(feat_oi)]

    tmp_acc <- eval_df(data_train_oi = tmp_df_train, 
                       data_test_oi = tmp_df_test)
    print(feat_oi)
    print(tmp_acc)
    if (tmp_acc >= acc_thresh){
      acc_thresh <- tmp_acc
      feat_to_drop <- feat_oi
    }
  }
  
  if(!is.na(feat_to_drop)){
    dropped_feat <- c(dropped_feat, feat_to_drop)
    final_df_test <- final_df_test[,!colnames(final_df_test) %in% dropped_feat]
    final_df_train <- final_df_train[,!colnames(final_df_train) %in% dropped_feat]
  }
}

##########################
# Forward selection
##########################

# here we will select one of our created feature in each of the two iterations, 
# that improve our model the most. 
for (iter in c(1,2)) {
  feat_to_select <- NA
  for (feat_oi in unselected_feat) {
    tmp_df_train <- final_df_train
    tmp_df_test <- final_df_test
    tmp_df_train[, feat_oi] <- feat_df_train[, feat_oi]
    tmp_df_test[, feat_oi] <- feat_df_test[, feat_oi]
    print(tmp_df_train[, feat_oi][1:5])
    tmp_acc <- eval_df(data_train_oi = tmp_df_train, 
                       data_test_oi = tmp_df_test)
    print(tmp_acc)
    if (tmp_acc >= acc_thresh){
      acc_thresh <- tmp_acc
      feat_to_select <- feat_oi
    }
  }
  if(!is.na(feat_to_select)){
    selected_feat <- c(selected_feat, feat_to_select)
    unselected_feat <- unselected_feat[-which(unselected_feat == feat_to_select)]
    final_df_test[, feat_to_select] <- feat_df_test[, feat_to_select]
    final_df_train[, feat_to_select] <- feat_df_train[, feat_to_select]
  }
}

# write.csv(final_df_test, "data/clean/test_feat_eng.csv", row.names = F)
# write.csv(final_df_train, "data/clean/train_feat_eng.csv", row.names = F)

Unsupervised Methode

Wir verwenden nun noch eine Unsupvised Learning Methode mit einem K-means Clustering, um einen Überblick zu bekommen und zu prüfen, ob die Daten mit einen K-Means CLustering gut zu clustern sind oder ob es keinen Sinn macht. Wir clustern die gesamten Daten mit einem K= 2, für die beiden Outputs (krank oder gesund). Wir werden eine Principal Component Analyse durchführen und anhand der ersten beiden Hauptkomponenten prüfen, ob sich dadurch ein sinnvolles Clustering ergibt. Dadurch erwarten wir uns noch einen etwas besseren Überblick über die Komplexität des Datensatzes.

clust_data <- data_clean
clust_data$Patient.age.quantile <- as.numeric(clust_data$Patient.age.quantile)
clust_data$Patient.addmited.to.regular.ward..1.yes..0.no. <- as.numeric(clust_data$Patient.addmited.to.regular.ward..1.yes..0.no.)
clust_data$Patient.addmited.to.semi.intensive.unit..1.yes..0.no. <- as.numeric(clust_data$Patient.addmited.to.semi.intensive.unit..1.yes..0.no.)
clust_data$Patient.addmited.to.intensive.care.unit..1.yes..0.no. <- as.numeric(clust_data$Patient.addmited.to.intensive.care.unit..1.yes..0.no.)
clust_data$sickness <- as.numeric(clust_data$sickness)

clust_data <- clust_data %>%
  select(-target)

kmeans <- kmeans(clust_data , centers=2, nstart = 10)
cluster_sizes <- kmeans$size
cluster_sizes

## [1] 312 220

Das Clustering ergibt deutlich balanciertere Daten als wir es in unserem Datensatz haben. Dies lässt bereits darauf schließen, dass ein K-Means CLustering wenig sinnvoll ist für diese Daten.

clust_pcov <- prcomp(clust_data, scale=T)
clust_pcov

## Standard deviations (1, .., p=16):
##  [1] 1.5406691 1.3356209 1.2616640 1.1186067 1.0553420 1.0478731 1.0304667
##  [8] 0.9876587 0.9507127 0.8656555 0.8164030 0.8055641 0.7411525 0.7217464
## [15] 0.6273475 0.5637475
## 
## Rotation (n x k) = (16 x 16):
##                                                               PC1          PC2
## Patient.age.quantile                                  -0.19413846  0.187449444
## Patient.addmited.to.regular.ward..1.yes..0.no.        -0.11065261  0.200066222
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no.  0.13550734  0.049507378
## Patient.addmited.to.intensive.care.unit..1.yes..0.no.  0.27786824  0.004503093
## sickness                                               0.16927561 -0.034312565
## Hematocrit                                            -0.23517886 -0.217172256
## Platelets                                              0.36640227 -0.165608793
## Mean.platelet.volume                                  -0.23751784  0.198702641
## Lymphocytes                                           -0.20421836  0.133540280
## Mean.corpuscular.hemoglobin.concentration..MCHC.      -0.20591512 -0.519981890
## Leukocytes                                             0.48156234 -0.197934987
## Basophils                                             -0.28860316  0.193215650
## Mean.corpuscular.hemoglobin..MCH.                     -0.28691907 -0.370110384
## Eosinophils                                           -0.06951921  0.087673222
## Monocytes                                             -0.25795570  0.019459641
## Red.blood.cell.distribution.width..RDW.                0.16391377  0.545291771
##                                                               PC3         PC4
## Patient.age.quantile                                  -0.29604962  0.46829930
## Patient.addmited.to.regular.ward..1.yes..0.no.        -0.27695513  0.29334910
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no. -0.21412471 -0.14232917
## Patient.addmited.to.intensive.care.unit..1.yes..0.no. -0.05310285  0.13022932
## sickness                                               0.21545327 -0.46293032
## Hematocrit                                             0.09525855  0.02619313
## Platelets                                              0.33877008  0.35048068
## Mean.platelet.volume                                  -0.19493342 -0.22389565
## Lymphocytes                                            0.43142098 -0.08987101
## Mean.corpuscular.hemoglobin.concentration..MCHC.      -0.10953932  0.03787172
## Leukocytes                                            -0.09743705  0.17946983
## Basophils                                              0.31817921  0.25383040
## Mean.corpuscular.hemoglobin..MCH.                     -0.09667817  0.20079521
## Eosinophils                                            0.50628362  0.27491373
## Monocytes                                              0.02963286 -0.21294148
## Red.blood.cell.distribution.width..RDW.               -0.05761826  0.04558037
##                                                               PC5         PC6
## Patient.age.quantile                                  -0.04702191 -0.26324880
## Patient.addmited.to.regular.ward..1.yes..0.no.         0.09750949  0.47077981
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no.  0.54870025 -0.47578755
## Patient.addmited.to.intensive.care.unit..1.yes..0.no. -0.20589955  0.10253999
## sickness                                              -0.18411684 -0.17999553
## Hematocrit                                            -0.54908860  0.03262305
## Platelets                                              0.07493366 -0.03965261
## Mean.platelet.volume                                  -0.34805248 -0.40708585
## Lymphocytes                                            0.30649224  0.12486890
## Mean.corpuscular.hemoglobin.concentration..MCHC.       0.15162020 -0.01471974
## Leukocytes                                            -0.19942928 -0.13282691
## Basophils                                             -0.08342793 -0.15990533
## Mean.corpuscular.hemoglobin..MCH.                      0.08200533 -0.29893038
## Eosinophils                                            0.02218811 -0.26255654
## Monocytes                                              0.08079865  0.20697350
## Red.blood.cell.distribution.width..RDW.               -0.07250742 -0.10615734
##                                                                PC7
## Patient.age.quantile                                  -0.246002221
## Patient.addmited.to.regular.ward..1.yes..0.no.         0.052326491
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no. -0.212853970
## Patient.addmited.to.intensive.care.unit..1.yes..0.no.  0.504724096
## sickness                                               0.007698463
## Hematocrit                                            -0.380935192
## Platelets                                             -0.191776483
## Mean.platelet.volume                                   0.241496891
## Lymphocytes                                            0.232670184
## Mean.corpuscular.hemoglobin.concentration..MCHC.       0.276274605
## Leukocytes                                            -0.085039378
## Basophils                                              0.133941487
## Mean.corpuscular.hemoglobin..MCH.                      0.207003553
## Eosinophils                                           -0.028286083
## Monocytes                                             -0.438143504
## Red.blood.cell.distribution.width..RDW.                0.071877480
##                                                                 PC8         PC9
## Patient.age.quantile                                   0.0008153688 -0.21397753
## Patient.addmited.to.regular.ward..1.yes..0.no.         0.2773029833  0.46945473
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no. -0.1456334286  0.03769581
## Patient.addmited.to.intensive.care.unit..1.yes..0.no. -0.0015993409 -0.56173561
## sickness                                               0.5361970886  0.19604898
## Hematocrit                                            -0.2859409128  0.05510906
## Platelets                                             -0.0674780516 -0.04045282
## Mean.platelet.volume                                  -0.1971974895  0.09335023
## Lymphocytes                                           -0.3863375713 -0.04918774
## Mean.corpuscular.hemoglobin.concentration..MCHC.       0.1076684460 -0.01713134
## Leukocytes                                             0.0102215046  0.12417809
## Basophils                                              0.1039656111  0.10748252
## Mean.corpuscular.hemoglobin..MCH.                      0.3157985674 -0.07131256
## Eosinophils                                            0.2386597879  0.05484507
## Monocytes                                              0.3412756274 -0.55357709
## Red.blood.cell.distribution.width..RDW.                0.2103859605 -0.14425127
##                                                              PC10        PC11
## Patient.age.quantile                                  -0.44573404  0.09690978
## Patient.addmited.to.regular.ward..1.yes..0.no.         0.03197212  0.12138688
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no.  0.21507492  0.42590957
## Patient.addmited.to.intensive.care.unit..1.yes..0.no.  0.04958867  0.41615198
## sickness                                              -0.35259674  0.28226539
## Hematocrit                                            -0.00293928  0.36596444
## Platelets                                             -0.02469238 -0.18878830
## Mean.platelet.volume                                   0.16137064 -0.37419368
## Lymphocytes                                           -0.34974919 -0.04557123
## Mean.corpuscular.hemoglobin.concentration..MCHC.       0.27257934 -0.04444997
## Leukocytes                                             0.15971779 -0.21861477
## Basophils                                              0.45577586  0.32460162
## Mean.corpuscular.hemoglobin..MCH.                     -0.30173545 -0.10022085
## Eosinophils                                            0.11580787 -0.10220747
## Monocytes                                              0.25408350 -0.17989946
## Red.blood.cell.distribution.width..RDW.                0.04147078 -0.14819267
##                                                              PC12         PC13
## Patient.age.quantile                                  -0.07044842  0.085030759
## Patient.addmited.to.regular.ward..1.yes..0.no.        -0.15045394  0.414497044
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no. -0.08429535  0.178395159
## Patient.addmited.to.intensive.care.unit..1.yes..0.no. -0.20362330  0.217562189
## sickness                                               0.03915194  0.225689016
## Hematocrit                                            -0.03332508  0.014972940
## Platelets                                              0.24923027  0.383961840
## Mean.platelet.volume                                  -0.11895083  0.453558552
## Lymphocytes                                            0.06219777  0.315948429
## Mean.corpuscular.hemoglobin.concentration..MCHC.      -0.07713866 -0.032444164
## Leukocytes                                             0.05943353  0.210863189
## Basophils                                              0.48092570  0.045259874
## Mean.corpuscular.hemoglobin..MCH.                      0.26043437  0.005039532
## Eosinophils                                           -0.68683446 -0.118737823
## Monocytes                                              0.01062471  0.325450043
## Red.blood.cell.distribution.width..RDW.                0.24617614 -0.264712078
##                                                               PC14        PC15
## Patient.age.quantile                                   0.008950547 -0.47063901
## Patient.addmited.to.regular.ward..1.yes..0.no.        -0.122060439  0.16866736
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no. -0.124919420  0.18772569
## Patient.addmited.to.intensive.care.unit..1.yes..0.no.  0.047318666  0.11447026
## sickness                                              -0.068483647 -0.22316528
## Hematocrit                                            -0.386476142  0.26570188
## Platelets                                             -0.007964199  0.09559330
## Mean.platelet.volume                                   0.061327270  0.05032694
## Lymphocytes                                           -0.317829645 -0.07581222
## Mean.corpuscular.hemoglobin.concentration..MCHC.      -0.499722251 -0.41828489
## Leukocytes                                            -0.154994722 -0.15873162
## Basophils                                              0.195982173 -0.20349589
## Mean.corpuscular.hemoglobin..MCH.                      0.031399264  0.53393072
## Eosinophils                                           -0.033604805  0.10017841
## Monocytes                                             -0.012146442  0.01337090
## Red.blood.cell.distribution.width..RDW.               -0.627587300  0.16438432
##                                                               PC16
## Patient.age.quantile                                  -0.023737582
## Patient.addmited.to.regular.ward..1.yes..0.no.        -0.016080647
## Patient.addmited.to.semi.intensive.unit..1.yes..0.no.  0.016267896
## Patient.addmited.to.intensive.care.unit..1.yes..0.no. -0.002161518
## sickness                                              -0.078646486
## Hematocrit                                            -0.007668320
## Platelets                                             -0.543664648
## Mean.platelet.volume                                  -0.175991648
## Lymphocytes                                            0.309940816
## Mean.corpuscular.hemoglobin.concentration..MCHC.      -0.233244091
## Leukocytes                                             0.660702469
## Basophils                                              0.120917437
## Mean.corpuscular.hemoglobin..MCH.                      0.179648384
## Eosinophils                                            0.067245302
## Monocytes                                              0.139017987
## Red.blood.cell.distribution.width..RDW.               -0.091563494

biplot(clust_pcov, main = "Biplot Princomp Method", expand = 1, col = c("blue", "red"))

library(factoextra)

## Welcome! Want to learn more? See two factoextra-related books at https://goo.gl/ve3WBa

fviz_cluster(kmeans, geom = "point", data = clust_pcov$x[,1:2]) + ggtitle(" K = 2 mit PCA")

Dieser Plot der Cluster zeigt deutlich, dass sich die einzelnen Punkte in den Clustern sehr stark überschneiden, deswegen ist es durch eine solche Cluster Methode nicht möglich zu prüfen ob ein Patient gesund oder mit Corona Infiziert ist.

Nach diesem ersten Überblick würden wir davon ausgehen, dass, auch auf Grund der Unbalanciertheit der Daten in Richtung nicht Corona infiziert, die Modelle sich leichter tun eine hohe Sensitivity zu erreichen, das erkennen von Corona Infizierten jedoch schwer werden könnte. Beginnen wir nun mit der entwicklung verschiedener Modelle zur Klassifizierung:

Support Vector Machines (SMVs)

In diesem Abschnitt wird versucht mit Hilfe von Support Vector Machines (SVMs) die Klassifizierung einer Corona Erkrankung zu verbessern.

plot_1 <- ggplot(data=data_train, aes(target)) + 
  geom_bar(stat = "count")
plot_1 <- plot_1 + ggtitle("Übersicht Klassen in Trainingsdatensatz") +
  xlab("Klassen") + ylab("Anzahl Observations pro Klasse") + scale_fill_brewer(palette="Dark2") + 
  theme(plot.title = element_text(hjust = 0.5))
plot_1

Der Trainingsdatensatz ist stark unbalanciert. Das heißt, das oben beschriebene, Rare Class Problem bei der Klassifizierung liegt definitv vor.

table(data_train$target)

## 
##   0   1 
## 380  47

#set.seed(1910837262)
#up_train_svm <- upSample(x = data_train[, -ncol(data_train)],
                     #y = data_train$target)                         
#table(up_train_svm$target) 

#up_train_svm <- up_train_svm %>%
    #select(-Class)
#print(str(up_train_svm))

Um eine bessere Trainingsgrundlage für das SVM zu haben, führen wir ein Upsampling der Trainingsdaten durch. Damit beheben wir das Rare Class Problem im Trainingsdatensatz. Nun haben wir im Trainingsdatensatz jeweils 380 Corona Infizierte und 380 Nicht Corona Infizierte. Allerdings werden diese Datensaätz aufgrund des Feature Engineerings nicht mehr benötigt.

up_train_svm <- read.csv("data/clean/train_feat_eng.csv")
data_test <- read.csv("data/clean/test_feat_eng.csv")

str(up_train_svm)

## 'data.frame':    760 obs. of  16 variables:
##  $ age              : int  17 1 9 11 13 9 17 17 19 10 ...
##  $ target           : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ reg_ward         : int  0 0 0 0 0 1 0 0 0 0 ...
##  $ semi_unit        : int  0 1 0 0 0 0 0 0 1 0 ...
##  $ intense_unit     : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ sickness         : int  1 0 1 1 1 1 1 1 1 0 ...
##  $ Hematocrit       : num  0.237 -1.572 -0.748 0.992 1.015 ...
##  $ Platelets        : num  -0.517 1.43 -0.429 0.073 -0.178 ...
##  $ Platelets_vol    : num  0.0107 -1.6722 -0.2137 -0.5503 0.796 ...
##  $ Lymphocytes      : num  0.31837 -0.00574 -1.11451 0.04544 -0.73071 ...
##  $ mean_hemoglobin  : num  -0.951 3.331 0.543 -0.453 -0.353 ...
##  $ Leukocytes       : num  -0.0946 0.3646 -0.8849 -0.2115 -0.0751 ...
##  $ Eosinophils      : num  1.482 1.019 -0.667 -0.709 0.218 ...
##  $ Monocytes        : num  0.3575 0.0687 1.2768 -0.2202 0.0687 ...
##  $ age_plat_leuk_eos: num  19.4 5.95 18.74 17.3 17.42 ...
##  $ age_leuk_eos     : num  10.06 4.28 9.86 9.91 9.56 ...

up_train_svm <- transform_type(up_train_svm)
data_test <- transform_type(data_test)

str(up_train_svm)

## 'data.frame':    760 obs. of  16 variables:
##  $ age              : num  17 1 9 11 13 9 17 17 19 10 ...
##  $ target           : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ reg_ward         : Factor w/ 2 levels "0","1": 1 1 1 1 1 2 1 1 1 1 ...
##  $ semi_unit        : Factor w/ 2 levels "0","1": 1 2 1 1 1 1 1 1 2 1 ...
##  $ intense_unit     : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ sickness         : Factor w/ 2 levels "0","1": 2 1 2 2 2 2 2 2 2 1 ...
##  $ Hematocrit       : num  0.237 -1.572 -0.748 0.992 1.015 ...
##  $ Platelets        : num  -0.517 1.43 -0.429 0.073 -0.178 ...
##  $ Platelets_vol    : num  0.0107 -1.6722 -0.2137 -0.5503 0.796 ...
##  $ Lymphocytes      : num  0.31837 -0.00574 -1.11451 0.04544 -0.73071 ...
##  $ mean_hemoglobin  : num  -0.951 3.331 0.543 -0.453 -0.353 ...
##  $ Leukocytes       : num  -0.0946 0.3646 -0.8849 -0.2115 -0.0751 ...
##  $ Eosinophils      : num  1.482 1.019 -0.667 -0.709 0.218 ...
##  $ Monocytes        : num  0.3575 0.0687 1.2768 -0.2202 0.0687 ...
##  $ age_plat_leuk_eos: num  19.4 5.95 18.74 17.3 17.42 ...
##  $ age_leuk_eos     : num  10.06 4.28 9.86 9.91 9.56 ...

methods = list("lssvmPoly", "lssvmRadial", "svmBoundrangeString", "svmRadialWeights", "svmExpoString", "svmLinear", "svmPoly", "svmRadial", "svmRadialCos", "svmRadialSigma", "svmSpectrumString")

set.seed(1910837388)

fitControl <- trainControl(## 10-fold CV
                           method = "repeatedcv",
                           number = 10,
                           ## repeated ten times
                           repeats = 3)

svm_fit_linear <- train(target ~ ., data = up_train_svm, 
                 method = "svmLinear", 
                 trControl = fitControl,
                 verbose = FALSE)
svm_fit_linear

## Support Vector Machines with Linear Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results:
## 
##   Accuracy   Kappa    
##   0.8587719  0.7175439
## 
## Tuning parameter 'C' was held constant at a value of 1

set.seed(1910837388)

svm_fit_linear <- train(target ~ ., data = up_train_svm, 
                 method = "svmLinear", 
                 trControl = fitControl,
                 tuneGrid = expand.grid(C = seq(0.000000001, 5, length = 50)),
                 verbose = FALSE)
svm_fit_linear

## Support Vector Machines with Linear Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results across tuning parameters:
## 
##   C            Accuracy   Kappa    
##   0.000000001  0.7385965  0.4771930
##   0.102040817  0.8609649  0.7219298
##   0.204081634  0.8605263  0.7210526
##   0.306122450  0.8592105  0.7184211
##   0.408163266  0.8574561  0.7149123
##   0.510204083  0.8565789  0.7131579
##   0.612244899  0.8570175  0.7140351
##   0.714285715  0.8570175  0.7140351
##   0.816326531  0.8583333  0.7166667
##   0.918367348  0.8587719  0.7175439
##   1.020408164  0.8587719  0.7175439
##   1.122448980  0.8583333  0.7166667
##   1.224489797  0.8587719  0.7175439
##   1.326530613  0.8587719  0.7175439
##   1.428571429  0.8592105  0.7184211
##   1.530612246  0.8587719  0.7175439
##   1.632653062  0.8587719  0.7175439
##   1.734693878  0.8592105  0.7184211
##   1.836734695  0.8592105  0.7184211
##   1.938775511  0.8583333  0.7166667
##   2.040816327  0.8583333  0.7166667
##   2.142857143  0.8587719  0.7175439
##   2.244897960  0.8587719  0.7175439
##   2.346938776  0.8587719  0.7175439
##   2.448979592  0.8587719  0.7175439
##   2.551020409  0.8587719  0.7175439
##   2.653061225  0.8587719  0.7175439
##   2.755102041  0.8587719  0.7175439
##   2.857142858  0.8587719  0.7175439
##   2.959183674  0.8587719  0.7175439
##   3.061224490  0.8587719  0.7175439
##   3.163265306  0.8587719  0.7175439
##   3.265306123  0.8592105  0.7184211
##   3.367346939  0.8592105  0.7184211
##   3.469387755  0.8592105  0.7184211
##   3.571428572  0.8592105  0.7184211
##   3.673469388  0.8592105  0.7184211
##   3.775510204  0.8592105  0.7184211
##   3.877551021  0.8592105  0.7184211
##   3.979591837  0.8592105  0.7184211
##   4.081632653  0.8592105  0.7184211
##   4.183673470  0.8592105  0.7184211
##   4.285714286  0.8592105  0.7184211
##   4.387755102  0.8592105  0.7184211
##   4.489795918  0.8592105  0.7184211
##   4.591836735  0.8592105  0.7184211
##   4.693877551  0.8592105  0.7184211
##   4.795918367  0.8592105  0.7184211
##   4.897959184  0.8592105  0.7184211
##   5.000000000  0.8592105  0.7184211
## 
## Accuracy was used to select the optimal model using the largest value.
## The final value used for the model was C = 0.1020408.

Optimierung des Tuning Parameters C, erst mit großer Range, dann im nächsten Schritt mit kleinere Range, aber auf die Ergebnisse des ersten Tests angepasst.

set.seed(1910837388)
svm_fit_linear <- train(target ~ ., data = up_train_svm, 
                 method = "svmLinear", 
                 trControl = fitControl,
                 tuneGrid = expand.grid(C = seq(0.01, 0.3, length = 50)),
                 verbose = FALSE)
svm_fit_linear

## Support Vector Machines with Linear Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results across tuning parameters:
## 
##   C           Accuracy   Kappa    
##   0.01000000  0.8526316  0.7052632
##   0.01591837  0.8548246  0.7096491
##   0.02183673  0.8557018  0.7114035
##   0.02775510  0.8592105  0.7184211
##   0.03367347  0.8574561  0.7149123
##   0.03959184  0.8596491  0.7192982
##   0.04551020  0.8631579  0.7263158
##   0.05142857  0.8640351  0.7280702
##   0.05734694  0.8640351  0.7280702
##   0.06326531  0.8622807  0.7245614
##   0.06918367  0.8640351  0.7280702
##   0.07510204  0.8627193  0.7254386
##   0.08102041  0.8618421  0.7236842
##   0.08693878  0.8609649  0.7219298
##   0.09285714  0.8605263  0.7210526
##   0.09877551  0.8609649  0.7219298
##   0.10469388  0.8618421  0.7236842
##   0.11061224  0.8622807  0.7245614
##   0.11653061  0.8618421  0.7236842
##   0.12244898  0.8618421  0.7236842
##   0.12836735  0.8618421  0.7236842
##   0.13428571  0.8622807  0.7245614
##   0.14020408  0.8618421  0.7236842
##   0.14612245  0.8622807  0.7245614
##   0.15204082  0.8618421  0.7236842
##   0.15795918  0.8614035  0.7228070
##   0.16387755  0.8614035  0.7228070
##   0.16979592  0.8605263  0.7210526
##   0.17571429  0.8605263  0.7210526
##   0.18163265  0.8605263  0.7210526
##   0.18755102  0.8600877  0.7201754
##   0.19346939  0.8600877  0.7201754
##   0.19938776  0.8605263  0.7210526
##   0.20530612  0.8600877  0.7201754
##   0.21122449  0.8600877  0.7201754
##   0.21714286  0.8600877  0.7201754
##   0.22306122  0.8600877  0.7201754
##   0.22897959  0.8605263  0.7210526
##   0.23489796  0.8600877  0.7201754
##   0.24081633  0.8605263  0.7210526
##   0.24673469  0.8605263  0.7210526
##   0.25265306  0.8605263  0.7210526
##   0.25857143  0.8605263  0.7210526
##   0.26448980  0.8600877  0.7201754
##   0.27040816  0.8600877  0.7201754
##   0.27632653  0.8600877  0.7201754
##   0.28224490  0.8600877  0.7201754
##   0.28816327  0.8592105  0.7184211
##   0.29408163  0.8592105  0.7184211
##   0.30000000  0.8592105  0.7184211
## 
## Accuracy was used to select the optimal model using the largest value.
## The final value used for the model was C = 0.05142857.

# Plot model accuracy vs different values of Cost
plot(svm_fit_linear)

Der Plot zeigt uns nochmal schön, was uns die Verherige Berechnung von ausgegeben hat. C = 0.0514 erzielt für das Modell die beste Accuracy auf den Trainingsdaten. Nun ermitteln wir die Accuracy für die Testdaten:

prediction_linear <- svm_fit_linear %>% predict(data_test)
mean(prediction_linear == data_test$target)

## [1] 0.8857143

Berechnen wir nun ein anderes, nicht lineares Modell, genau wie davor erstmal ohne Tuning Parameter, um uns dann anzunähern:

set.seed(1910837388)

svm_fit_radial <- train(target ~ ., data = up_train_svm, 
                 method = "svmRadial", 
                 tuneLength = 9,
                 trControl = fitControl,
                 verbose = FALSE)
svm_fit_radial

## Support Vector Machines with Radial Basis Function Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results across tuning parameters:
## 
##   C      Accuracy   Kappa    
##    0.25  0.8912281  0.7824561
##    0.50  0.9109649  0.8219298
##    1.00  0.9228070  0.8456140
##    2.00  0.9254386  0.8508772
##    4.00  0.9276316  0.8552632
##    8.00  0.9276316  0.8552632
##   16.00  0.9364035  0.8728070
##   32.00  0.9346491  0.8692982
##   64.00  0.9337719  0.8675439
## 
## Tuning parameter 'sigma' was held constant at a value of 0.05561306
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were sigma = 0.05561306 and C = 16.

set.seed(1910837388)
# Use the expand.grid to specify the search space   
grid <- expand.grid(sigma = seq(0.01, 0.1, length = 10),
                    C = seq(14, 18, length = 20))

svm_fit_radial <- train(target ~ ., data = up_train_svm, 
                 method = "svmRadial", 
                 tuneGrid = grid,
                 trControl = fitControl,
                 verbose = FALSE)
svm_fit_radial

## Support Vector Machines with Radial Basis Function Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results across tuning parameters:
## 
##   sigma  C         Accuracy   Kappa    
##   0.01   14.00000  0.9087719  0.8175439
##   0.01   14.21053  0.9100877  0.8201754
##   0.01   14.42105  0.9105263  0.8210526
##   0.01   14.63158  0.9105263  0.8210526
##   0.01   14.84211  0.9100877  0.8201754
##   0.01   15.05263  0.9100877  0.8201754
##   0.01   15.26316  0.9096491  0.8192982
##   0.01   15.47368  0.9096491  0.8192982
##   0.01   15.68421  0.9096491  0.8192982
##   0.01   15.89474  0.9100877  0.8201754
##   0.01   16.10526  0.9100877  0.8201754
##   0.01   16.31579  0.9109649  0.8219298
##   0.01   16.52632  0.9109649  0.8219298
##   0.01   16.73684  0.9105263  0.8210526
##   0.01   16.94737  0.9118421  0.8236842
##   0.01   17.15789  0.9118421  0.8236842
##   0.01   17.36842  0.9131579  0.8263158
##   0.01   17.57895  0.9131579  0.8263158
##   0.01   17.78947  0.9131579  0.8263158
##   0.01   18.00000  0.9135965  0.8271930
##   0.02   14.00000  0.9258772  0.8517544
##   0.02   14.21053  0.9258772  0.8517544
##   0.02   14.42105  0.9258772  0.8517544
##   0.02   14.63158  0.9258772  0.8517544
##   0.02   14.84211  0.9263158  0.8526316
##   0.02   15.05263  0.9258772  0.8517544
##   0.02   15.26316  0.9254386  0.8508772
##   0.02   15.47368  0.9254386  0.8508772
##   0.02   15.68421  0.9254386  0.8508772
##   0.02   15.89474  0.9258772  0.8517544
##   0.02   16.10526  0.9263158  0.8526316
##   0.02   16.31579  0.9263158  0.8526316
##   0.02   16.52632  0.9263158  0.8526316
##   0.02   16.73684  0.9267544  0.8535088
##   0.02   16.94737  0.9263158  0.8526316
##   0.02   17.15789  0.9258772  0.8517544
##   0.02   17.36842  0.9258772  0.8517544
##   0.02   17.57895  0.9258772  0.8517544
##   0.02   17.78947  0.9258772  0.8517544
##   0.02   18.00000  0.9258772  0.8517544
##   0.03   14.00000  0.9293860  0.8587719
##   0.03   14.21053  0.9293860  0.8587719
##   0.03   14.42105  0.9298246  0.8596491
##   0.03   14.63158  0.9293860  0.8587719
##   0.03   14.84211  0.9293860  0.8587719
##   0.03   15.05263  0.9293860  0.8587719
##   0.03   15.26316  0.9293860  0.8587719
##   0.03   15.47368  0.9289474  0.8578947
##   0.03   15.68421  0.9280702  0.8561404
##   0.03   15.89474  0.9271930  0.8543860
##   0.03   16.10526  0.9267544  0.8535088
##   0.03   16.31579  0.9267544  0.8535088
##   0.03   16.52632  0.9263158  0.8526316
##   0.03   16.73684  0.9267544  0.8535088
##   0.03   16.94737  0.9254386  0.8508772
##   0.03   17.15789  0.9254386  0.8508772
##   0.03   17.36842  0.9254386  0.8508772
##   0.03   17.57895  0.9254386  0.8508772
##   0.03   17.78947  0.9254386  0.8508772
##   0.03   18.00000  0.9258772  0.8517544
##   0.04   14.00000  0.9267544  0.8535088
##   0.04   14.21053  0.9267544  0.8535088
##   0.04   14.42105  0.9267544  0.8535088
##   0.04   14.63158  0.9263158  0.8526316
##   0.04   14.84211  0.9271930  0.8543860
##   0.04   15.05263  0.9280702  0.8561404
##   0.04   15.26316  0.9280702  0.8561404
##   0.04   15.47368  0.9280702  0.8561404
##   0.04   15.68421  0.9280702  0.8561404
##   0.04   15.89474  0.9280702  0.8561404
##   0.04   16.10526  0.9289474  0.8578947
##   0.04   16.31579  0.9293860  0.8587719
##   0.04   16.52632  0.9289474  0.8578947
##   0.04   16.73684  0.9285088  0.8570175
##   0.04   16.94737  0.9293860  0.8587719
##   0.04   17.15789  0.9293860  0.8587719
##   0.04   17.36842  0.9298246  0.8596491
##   0.04   17.57895  0.9293860  0.8587719
##   0.04   17.78947  0.9293860  0.8587719
##   0.04   18.00000  0.9298246  0.8596491
##   0.05   14.00000  0.9311404  0.8622807
##   0.05   14.21053  0.9311404  0.8622807
##   0.05   14.42105  0.9311404  0.8622807
##   0.05   14.63158  0.9315789  0.8631579
##   0.05   14.84211  0.9320175  0.8640351
##   0.05   15.05263  0.9320175  0.8640351
##   0.05   15.26316  0.9320175  0.8640351
##   0.05   15.47368  0.9320175  0.8640351
##   0.05   15.68421  0.9333333  0.8666667
##   0.05   15.89474  0.9342105  0.8684211
##   0.05   16.10526  0.9350877  0.8701754
##   0.05   16.31579  0.9350877  0.8701754
##   0.05   16.52632  0.9355263  0.8710526
##   0.05   16.73684  0.9355263  0.8710526
##   0.05   16.94737  0.9355263  0.8710526
##   0.05   17.15789  0.9355263  0.8710526
##   0.05   17.36842  0.9350877  0.8701754
##   0.05   17.57895  0.9346491  0.8692982
##   0.05   17.78947  0.9346491  0.8692982
##   0.05   18.00000  0.9342105  0.8684211
##   0.06   14.00000  0.9364035  0.8728070
##   0.06   14.21053  0.9372807  0.8745614
##   0.06   14.42105  0.9359649  0.8719298
##   0.06   14.63158  0.9359649  0.8719298
##   0.06   14.84211  0.9359649  0.8719298
##   0.06   15.05263  0.9364035  0.8728070
##   0.06   15.26316  0.9364035  0.8728070
##   0.06   15.47368  0.9364035  0.8728070
##   0.06   15.68421  0.9364035  0.8728070
##   0.06   15.89474  0.9368421  0.8736842
##   0.06   16.10526  0.9372807  0.8745614
##   0.06   16.31579  0.9381579  0.8763158
##   0.06   16.52632  0.9385965  0.8771930
##   0.06   16.73684  0.9385965  0.8771930
##   0.06   16.94737  0.9385965  0.8771930
##   0.06   17.15789  0.9390351  0.8780702
##   0.06   17.36842  0.9390351  0.8780702
##   0.06   17.57895  0.9385965  0.8771930
##   0.06   17.78947  0.9390351  0.8780702
##   0.06   18.00000  0.9390351  0.8780702
##   0.07   14.00000  0.9377193  0.8754386
##   0.07   14.21053  0.9377193  0.8754386
##   0.07   14.42105  0.9385965  0.8771930
##   0.07   14.63158  0.9385965  0.8771930
##   0.07   14.84211  0.9385965  0.8771930
##   0.07   15.05263  0.9381579  0.8763158
##   0.07   15.26316  0.9381579  0.8763158
##   0.07   15.47368  0.9381579  0.8763158
##   0.07   15.68421  0.9377193  0.8754386
##   0.07   15.89474  0.9377193  0.8754386
##   0.07   16.10526  0.9372807  0.8745614
##   0.07   16.31579  0.9372807  0.8745614
##   0.07   16.52632  0.9368421  0.8736842
##   0.07   16.73684  0.9368421  0.8736842
##   0.07   16.94737  0.9359649  0.8719298
##   0.07   17.15789  0.9359649  0.8719298
##   0.07   17.36842  0.9355263  0.8710526
##   0.07   17.57895  0.9355263  0.8710526
##   0.07   17.78947  0.9355263  0.8710526
##   0.07   18.00000  0.9355263  0.8710526
##   0.08   14.00000  0.9355263  0.8710526
##   0.08   14.21053  0.9355263  0.8710526
##   0.08   14.42105  0.9355263  0.8710526
##   0.08   14.63158  0.9350877  0.8701754
##   0.08   14.84211  0.9350877  0.8701754
##   0.08   15.05263  0.9350877  0.8701754
##   0.08   15.26316  0.9350877  0.8701754
##   0.08   15.47368  0.9346491  0.8692982
##   0.08   15.68421  0.9342105  0.8684211
##   0.08   15.89474  0.9342105  0.8684211
##   0.08   16.10526  0.9346491  0.8692982
##   0.08   16.31579  0.9346491  0.8692982
##   0.08   16.52632  0.9346491  0.8692982
##   0.08   16.73684  0.9346491  0.8692982
##   0.08   16.94737  0.9346491  0.8692982
##   0.08   17.15789  0.9346491  0.8692982
##   0.08   17.36842  0.9346491  0.8692982
##   0.08   17.57895  0.9350877  0.8701754
##   0.08   17.78947  0.9350877  0.8701754
##   0.08   18.00000  0.9346491  0.8692982
##   0.09   14.00000  0.9346491  0.8692982
##   0.09   14.21053  0.9346491  0.8692982
##   0.09   14.42105  0.9346491  0.8692982
##   0.09   14.63158  0.9342105  0.8684211
##   0.09   14.84211  0.9342105  0.8684211
##   0.09   15.05263  0.9346491  0.8692982
##   0.09   15.26316  0.9342105  0.8684211
##   0.09   15.47368  0.9346491  0.8692982
##   0.09   15.68421  0.9350877  0.8701754
##   0.09   15.89474  0.9346491  0.8692982
##   0.09   16.10526  0.9350877  0.8701754
##   0.09   16.31579  0.9350877  0.8701754
##   0.09   16.52632  0.9350877  0.8701754
##   0.09   16.73684  0.9350877  0.8701754
##   0.09   16.94737  0.9350877  0.8701754
##   0.09   17.15789  0.9350877  0.8701754
##   0.09   17.36842  0.9350877  0.8701754
##   0.09   17.57895  0.9350877  0.8701754
##   0.09   17.78947  0.9355263  0.8710526
##   0.09   18.00000  0.9355263  0.8710526
##   0.10   14.00000  0.9359649  0.8719298
##   0.10   14.21053  0.9359649  0.8719298
##   0.10   14.42105  0.9359649  0.8719298
##   0.10   14.63158  0.9359649  0.8719298
##   0.10   14.84211  0.9359649  0.8719298
##   0.10   15.05263  0.9359649  0.8719298
##   0.10   15.26316  0.9359649  0.8719298
##   0.10   15.47368  0.9359649  0.8719298
##   0.10   15.68421  0.9355263  0.8710526
##   0.10   15.89474  0.9355263  0.8710526
##   0.10   16.10526  0.9355263  0.8710526
##   0.10   16.31579  0.9355263  0.8710526
##   0.10   16.52632  0.9355263  0.8710526
##   0.10   16.73684  0.9355263  0.8710526
##   0.10   16.94737  0.9355263  0.8710526
##   0.10   17.15789  0.9355263  0.8710526
##   0.10   17.36842  0.9359649  0.8719298
##   0.10   17.57895  0.9364035  0.8728070
##   0.10   17.78947  0.9364035  0.8728070
##   0.10   18.00000  0.9364035  0.8728070
## 
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were sigma = 0.06 and C = 17.15789.

# Plot model accuracy vs different values of Cost
plot(svm_fit_radial)

prediction_radial <- svm_fit_radial %>% predict(data_test)
mean(prediction_radial == data_test$target)

## [1] 0.9047619

set.seed(1910837388)

svm_fit_poly <- train(target ~ ., data = up_train_svm, 
                 method = "svmPoly", 
                 tuneLength = 4,
                 trControl = fitControl,
                 verbose = FALSE)
svm_fit_poly

## Support Vector Machines with Polynomial Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results across tuning parameters:
## 
##   degree  scale  C     Accuracy   Kappa    
##   1       0.001  0.25  0.7815789  0.5631579
##   1       0.001  0.50  0.8245614  0.6491228
##   1       0.001  1.00  0.8359649  0.6719298
##   1       0.001  2.00  0.8394737  0.6789474
##   1       0.010  0.25  0.8416667  0.6833333
##   1       0.010  0.50  0.8478070  0.6956140
##   1       0.010  1.00  0.8526316  0.7052632
##   1       0.010  2.00  0.8561404  0.7122807
##   1       0.100  0.25  0.8557018  0.7114035
##   1       0.100  0.50  0.8640351  0.7280702
##   1       0.100  1.00  0.8614035  0.7228070
##   1       0.100  2.00  0.8605263  0.7210526
##   1       1.000  0.25  0.8605263  0.7210526
##   1       1.000  0.50  0.8565789  0.7131579
##   1       1.000  1.00  0.8587719  0.7175439
##   1       1.000  2.00  0.8587719  0.7175439
##   2       0.001  0.25  0.8245614  0.6491228
##   2       0.001  0.50  0.8359649  0.6719298
##   2       0.001  1.00  0.8403509  0.6807018
##   2       0.001  2.00  0.8434211  0.6868421
##   2       0.010  0.25  0.8482456  0.6964912
##   2       0.010  0.50  0.8561404  0.7122807
##   2       0.010  1.00  0.8640351  0.7280702
##   2       0.010  2.00  0.8728070  0.7456140
##   2       0.100  0.25  0.9109649  0.8219298
##   2       0.100  0.50  0.9157895  0.8315789
##   2       0.100  1.00  0.9236842  0.8473684
##   2       0.100  2.00  0.9254386  0.8508772
##   2       1.000  0.25  0.9377193  0.8754386
##   2       1.000  0.50  0.9280702  0.8561404
##   2       1.000  1.00  0.9241228  0.8482456
##   2       1.000  2.00  0.9241228  0.8482456
##   3       0.001  0.25  0.8280702  0.6561404
##   3       0.001  0.50  0.8429825  0.6859649
##   3       0.001  1.00  0.8403509  0.6807018
##   3       0.001  2.00  0.8460526  0.6921053
##   3       0.010  0.25  0.8583333  0.7166667
##   3       0.010  0.50  0.8640351  0.7280702
##   3       0.010  1.00  0.8811404  0.7622807
##   3       0.010  2.00  0.8864035  0.7728070
##   3       0.100  0.25  0.9228070  0.8456140
##   3       0.100  0.50  0.9271930  0.8543860
##   3       0.100  1.00  0.9250000  0.8500000
##   3       0.100  2.00  0.9223684  0.8447368
##   3       1.000  0.25  0.9122807  0.8245614
##   3       1.000  0.50  0.9149123  0.8298246
##   3       1.000  1.00  0.9162281  0.8324561
##   3       1.000  2.00  0.9166667  0.8333333
## 
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were degree = 2, scale = 1 and C = 0.25.

The final values used for the model were degree = 2, scale = 1 and C = 1.

#set.seed(1910837388)

#grid <- expand.grid(degree = seq(1, 4, length = 4),
                    #scale = seq(0.5, 2, length = 10),
                    #C = seq(0.5, 2, length = 10))

#svm_fit_poly <- train(target ~ ., data = up_train_svm, 
                 #method = "svmPoly", 
                 #tuneGrid = grid,
                 #trControl = fitControl,
                 #verbose = FALSE)
#svm_fit_poly

The final values used for the model were degree = 2, scale = 0.66667 and C = 0.5.

set.seed(1910837388)

grid <- expand.grid(degree = seq(2, 2, length = 1),
                    scale = seq(0.5, 0.7, length = 10),
                    C = seq(0.4, 0.6, length = 10))

svm_fit_poly <- train(target ~ ., data = up_train_svm, 
                 method = "svmPoly", 
                 tuneGrid = grid,
                 preProc = c("center","scale"),
                 trControl = fitControl,
                 verbose = FALSE)
svm_fit_poly

## Support Vector Machines with Polynomial Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## Pre-processing: centered (15), scaled (15) 
## Resampling: Cross-Validated (10 fold, repeated 3 times) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results across tuning parameters:
## 
##   scale      C          Accuracy   Kappa    
##   0.5000000  0.4000000  0.9298246  0.8596491
##   0.5000000  0.4222222  0.9302632  0.8605263
##   0.5000000  0.4444444  0.9293860  0.8587719
##   0.5000000  0.4666667  0.9311404  0.8622807
##   0.5000000  0.4888889  0.9324561  0.8649123
##   0.5000000  0.5111111  0.9320175  0.8640351
##   0.5000000  0.5333333  0.9350877  0.8701754
##   0.5000000  0.5555556  0.9320175  0.8640351
##   0.5000000  0.5777778  0.9320175  0.8640351
##   0.5000000  0.6000000  0.9333333  0.8666667
##   0.5222222  0.4000000  0.9298246  0.8596491
##   0.5222222  0.4222222  0.9302632  0.8605263
##   0.5222222  0.4444444  0.9324561  0.8649123
##   0.5222222  0.4666667  0.9320175  0.8640351
##   0.5222222  0.4888889  0.9346491  0.8692982
##   0.5222222  0.5111111  0.9315789  0.8631579
##   0.5222222  0.5333333  0.9324561  0.8649123
##   0.5222222  0.5555556  0.9342105  0.8684211
##   0.5222222  0.5777778  0.9337719  0.8675439
##   0.5222222  0.6000000  0.9337719  0.8675439
##   0.5444444  0.4000000  0.9324561  0.8649123
##   0.5444444  0.4222222  0.9320175  0.8640351
##   0.5444444  0.4444444  0.9346491  0.8692982
##   0.5444444  0.4666667  0.9311404  0.8622807
##   0.5444444  0.4888889  0.9320175  0.8640351
##   0.5444444  0.5111111  0.9337719  0.8675439
##   0.5444444  0.5333333  0.9342105  0.8684211
##   0.5444444  0.5555556  0.9342105  0.8684211
##   0.5444444  0.5777778  0.9337719  0.8675439
##   0.5444444  0.6000000  0.9350877  0.8701754
##   0.5666667  0.4000000  0.9324561  0.8649123
##   0.5666667  0.4222222  0.9342105  0.8684211
##   0.5666667  0.4444444  0.9311404  0.8622807
##   0.5666667  0.4666667  0.9333333  0.8666667
##   0.5666667  0.4888889  0.9346491  0.8692982
##   0.5666667  0.5111111  0.9342105  0.8684211
##   0.5666667  0.5333333  0.9337719  0.8675439
##   0.5666667  0.5555556  0.9350877  0.8701754
##   0.5666667  0.5777778  0.9364035  0.8728070
##   0.5666667  0.6000000  0.9372807  0.8745614
##   0.5888889  0.4000000  0.9320175  0.8640351
##   0.5888889  0.4222222  0.9324561  0.8649123
##   0.5888889  0.4444444  0.9337719  0.8675439
##   0.5888889  0.4666667  0.9342105  0.8684211
##   0.5888889  0.4888889  0.9333333  0.8666667
##   0.5888889  0.5111111  0.9346491  0.8692982
##   0.5888889  0.5333333  0.9364035  0.8728070
##   0.5888889  0.5555556  0.9368421  0.8736842
##   0.5888889  0.5777778  0.9385965  0.8771930
##   0.5888889  0.6000000  0.9385965  0.8771930
##   0.6111111  0.4000000  0.9328947  0.8657895
##   0.6111111  0.4222222  0.9342105  0.8684211
##   0.6111111  0.4444444  0.9346491  0.8692982
##   0.6111111  0.4666667  0.9342105  0.8684211
##   0.6111111  0.4888889  0.9355263  0.8710526
##   0.6111111  0.5111111  0.9372807  0.8745614
##   0.6111111  0.5333333  0.9385965  0.8771930
##   0.6111111  0.5555556  0.9381579  0.8763158
##   0.6111111  0.5777778  0.9385965  0.8771930
##   0.6111111  0.6000000  0.9390351  0.8780702
##   0.6333333  0.4000000  0.9342105  0.8684211
##   0.6333333  0.4222222  0.9346491  0.8692982
##   0.6333333  0.4444444  0.9346491  0.8692982
##   0.6333333  0.4666667  0.9364035  0.8728070
##   0.6333333  0.4888889  0.9377193  0.8754386
##   0.6333333  0.5111111  0.9381579  0.8763158
##   0.6333333  0.5333333  0.9390351  0.8780702
##   0.6333333  0.5555556  0.9390351  0.8780702
##   0.6333333  0.5777778  0.9394737  0.8789474
##   0.6333333  0.6000000  0.9399123  0.8798246
##   0.6555556  0.4000000  0.9337719  0.8675439
##   0.6555556  0.4222222  0.9350877  0.8701754
##   0.6555556  0.4444444  0.9372807  0.8745614
##   0.6555556  0.4666667  0.9385965  0.8771930
##   0.6555556  0.4888889  0.9381579  0.8763158
##   0.6555556  0.5111111  0.9390351  0.8780702
##   0.6555556  0.5333333  0.9394737  0.8789474
##   0.6555556  0.5555556  0.9385965  0.8771930
##   0.6555556  0.5777778  0.9390351  0.8780702
##   0.6555556  0.6000000  0.9377193  0.8754386
##   0.6777778  0.4000000  0.9355263  0.8710526
##   0.6777778  0.4222222  0.9368421  0.8736842
##   0.6777778  0.4444444  0.9385965  0.8771930
##   0.6777778  0.4666667  0.9385965  0.8771930
##   0.6777778  0.4888889  0.9394737  0.8789474
##   0.6777778  0.5111111  0.9390351  0.8780702
##   0.6777778  0.5333333  0.9390351  0.8780702
##   0.6777778  0.5555556  0.9385965  0.8771930
##   0.6777778  0.5777778  0.9372807  0.8745614
##   0.6777778  0.6000000  0.9359649  0.8719298
##   0.7000000  0.4000000  0.9368421  0.8736842
##   0.7000000  0.4222222  0.9381579  0.8763158
##   0.7000000  0.4444444  0.9390351  0.8780702
##   0.7000000  0.4666667  0.9394737  0.8789474
##   0.7000000  0.4888889  0.9390351  0.8780702
##   0.7000000  0.5111111  0.9390351  0.8780702
##   0.7000000  0.5333333  0.9377193  0.8754386
##   0.7000000  0.5555556  0.9364035  0.8728070
##   0.7000000  0.5777778  0.9359649  0.8719298
##   0.7000000  0.6000000  0.9355263  0.8710526
## 
## Tuning parameter 'degree' was held constant at a value of 2
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were degree = 2, scale = 0.6333333 and C
##  = 0.6.

The final values used for the model were degree = 2, scale = 0.633333 and C = 0.6.

plot(svm_fit_poly)

prediction_poly <- svm_fit_poly %>% predict(data_test)
mean(prediction_poly == data_test$target)

## [1] 0.8761905

set.seed(1910837388)

svm_fit_RSigma <- train(target ~ ., data = up_train_svm, 
                 method = "svmRadialSigma",
                 trControl = trainControl(method = "cv"),
                 verbose = FALSE)
svm_fit_RSigma

## Support Vector Machines with Radial Basis Function Kernel 
## 
## 760 samples
##  15 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 684, 684, 684, 684, 684, 684, ... 
## Resampling results across tuning parameters:
## 
##   sigma       C     Accuracy   Kappa    
##   0.01930794  0.25  0.8605263  0.7210526
##   0.01930794  0.50  0.8605263  0.7210526
##   0.01930794  1.00  0.8842105  0.7684211
##   0.05561306  0.25  0.8894737  0.7789474
##   0.05561306  0.50  0.9078947  0.8157895
##   0.05561306  1.00  0.9210526  0.8421053
##   0.09191819  0.25  0.9118421  0.8236842
##   0.09191819  0.50  0.9250000  0.8500000
##   0.09191819  1.00  0.9236842  0.8473684
## 
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were sigma = 0.09191819 and C = 0.5.

prediction_RSigma <- svm_fit_RSigma %>% predict(data_test)
mean(prediction_RSigma == data_test$target)

## [1] 0.9714286

plot(svm_fit_RSigma)

results_svm_linear <- data.frame(actual = data_test$target, prediction = prediction_linear)
results_svm_radial <- data.frame(actual = data_test$target, prediction = prediction_radial)
results_svm_poly <- data.frame(actual = data_test$target, prediction = prediction_poly)
results_svm_RS <- data.frame(actual = data_test$target, prediction = prediction_RSigma)
CM_linear <- confusionMatrix(table(results_svm_linear$actual,results_svm_linear$prediction))
CM_radial <- confusionMatrix(table(results_svm_radial$actual,results_svm_radial$prediction))
CM_poly <- confusionMatrix(table(results_svm_poly$actual,results_svm_poly$prediction))
CM_RS <- confusionMatrix(table(results_svm_RS$actual,results_svm_RS$prediction))
CM_linear

## Confusion Matrix and Statistics
## 
##    
##      0  1
##   0 84 10
##   1  2  9
##                                           
##                Accuracy : 0.8857          
##                  95% CI : (0.8089, 0.9395)
##     No Information Rate : 0.819           
##     P-Value [Acc > NIR] : 0.04408         
##                                           
##                   Kappa : 0.5388          
##                                           
##  Mcnemar's Test P-Value : 0.04331         
##                                           
##             Sensitivity : 0.9767          
##             Specificity : 0.4737          
##          Pos Pred Value : 0.8936          
##          Neg Pred Value : 0.8182          
##              Prevalence : 0.8190          
##          Detection Rate : 0.8000          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.7252          
##                                           
##        'Positive' Class : 0               
## 

CM_radial

## Confusion Matrix and Statistics
## 
##    
##      0  1
##   0 86  8
##   1  2  9
##                                           
##                Accuracy : 0.9048          
##                  95% CI : (0.8318, 0.9534)
##     No Information Rate : 0.8381          
##     P-Value [Acc > NIR] : 0.0362          
##                                           
##                   Kappa : 0.5908          
##                                           
##  Mcnemar's Test P-Value : 0.1138          
##                                           
##             Sensitivity : 0.9773          
##             Specificity : 0.5294          
##          Pos Pred Value : 0.9149          
##          Neg Pred Value : 0.8182          
##              Prevalence : 0.8381          
##          Detection Rate : 0.8190          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.7533          
##                                           
##        'Positive' Class : 0               
## 

CM_poly

## Confusion Matrix and Statistics
## 
##    
##      0  1
##   0 84 10
##   1  3  8
##                                           
##                Accuracy : 0.8762          
##                  95% CI : (0.7976, 0.9324)
##     No Information Rate : 0.8286          
##     P-Value [Acc > NIR] : 0.11937         
##                                           
##                   Kappa : 0.4847          
##                                           
##  Mcnemar's Test P-Value : 0.09609         
##                                           
##             Sensitivity : 0.9655          
##             Specificity : 0.4444          
##          Pos Pred Value : 0.8936          
##          Neg Pred Value : 0.7273          
##              Prevalence : 0.8286          
##          Detection Rate : 0.8000          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.7050          
##                                           
##        'Positive' Class : 0               
## 

CM_RS

## Confusion Matrix and Statistics
## 
##    
##      0  1
##   0 93  1
##   1  2  9
##                                           
##                Accuracy : 0.9714          
##                  95% CI : (0.9188, 0.9941)
##     No Information Rate : 0.9048          
##     P-Value [Acc > NIR] : 0.007949        
##                                           
##                   Kappa : 0.8413          
##                                           
##  Mcnemar's Test P-Value : 1.000000        
##                                           
##             Sensitivity : 0.9789          
##             Specificity : 0.9000          
##          Pos Pred Value : 0.9894          
##          Neg Pred Value : 0.8182          
##              Prevalence : 0.9048          
##          Detection Rate : 0.8857          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.9395          
##                                           
##        'Positive' Class : 0               
## 

Wenn wir uns die Confusion Matrizen der finalen Kernel mit optimierten Parametern anschauen, dann fällt uns auf, dass, unabhängig vom Kernel, die Specifity sehr niedrig ist, außer bei dem Radial Sigma Kernel und somit viele Positive Erkrankte nicht als solche erkannt werden. Im Vergleich mit den anderen Kernel scheidet der Radial Sigma Kernel jedoch deutlich besser ab, sowohl bei Accuracy, als auch bei Sensivity und Specivity. Schauen wir uns im nächsten Schritt noch eine übersichtlichere Tabelle an:

results <- resamples(list(Linear=svm_fit_linear, Radial=svm_fit_radial, Polynomial=svm_fit_poly))
results$values

##       Resample Linear~Accuracy Linear~Kappa Radial~Accuracy Radial~Kappa
## 1  Fold01.Rep1       0.8947368    0.7894737       0.9605263    0.9210526
## 2  Fold01.Rep2       0.9210526    0.8421053       0.9605263    0.9210526
## 3  Fold01.Rep3       0.8421053    0.6842105       0.9210526    0.8421053
## 4  Fold02.Rep1       0.8552632    0.7105263       0.9210526    0.8421053
## 5  Fold02.Rep2       0.8815789    0.7631579       0.9605263    0.9210526
## 6  Fold02.Rep3       0.9210526    0.8421053       0.9736842    0.9473684
## 7  Fold03.Rep1       0.9078947    0.8157895       0.9605263    0.9210526
## 8  Fold03.Rep2       0.8421053    0.6842105       0.9078947    0.8157895
## 9  Fold03.Rep3       0.9078947    0.8157895       0.9736842    0.9473684
## 10 Fold04.Rep1       0.8552632    0.7105263       0.8289474    0.6578947
## 11 Fold04.Rep2       0.8815789    0.7631579       0.9210526    0.8421053
## 12 Fold04.Rep3       0.7894737    0.5789474       0.8815789    0.7631579
## 13 Fold05.Rep1       0.8289474    0.6578947       0.9078947    0.8157895
## 14 Fold05.Rep2       0.9473684    0.8947368       0.9473684    0.8947368
## 15 Fold05.Rep3       0.8947368    0.7894737       0.9342105    0.8684211
## 16 Fold06.Rep1       0.8815789    0.7631579       0.9736842    0.9473684
## 17 Fold06.Rep2       0.7763158    0.5526316       0.9342105    0.8684211
## 18 Fold06.Rep3       0.7894737    0.5789474       0.9210526    0.8421053
## 19 Fold07.Rep1       0.9078947    0.8157895       1.0000000    1.0000000
## 20 Fold07.Rep2       0.8421053    0.6842105       0.9605263    0.9210526
## 21 Fold07.Rep3       0.8289474    0.6578947       0.9210526    0.8421053
## 22 Fold08.Rep1       0.9078947    0.8157895       0.9210526    0.8421053
## 23 Fold08.Rep2       0.8289474    0.6578947       0.9473684    0.8947368
## 24 Fold08.Rep3       0.8815789    0.7631579       0.9868421    0.9736842
## 25 Fold09.Rep1       0.8157895    0.6315789       0.9868421    0.9736842
## 26 Fold09.Rep2       0.8947368    0.7894737       0.9342105    0.8684211
## 27 Fold09.Rep3       0.9078947    0.8157895       0.9736842    0.9473684
## 28 Fold10.Rep1       0.8157895    0.6315789       0.9473684    0.8947368
## 29 Fold10.Rep2       0.8026316    0.6052632       0.8815789    0.7631579
## 30 Fold10.Rep3       0.8684211    0.7368421       0.9210526    0.8421053
##    Polynomial~Accuracy Polynomial~Kappa
## 1            0.9605263        0.9210526
## 2            0.9605263        0.9210526
## 3            0.8947368        0.7894737
## 4            0.9342105        0.8684211
## 5            0.9868421        0.9736842
## 6            0.9736842        0.9473684
## 7            0.9342105        0.8684211
## 8            0.9473684        0.8947368
## 9            0.9736842        0.9473684
## 10           0.8421053        0.6842105
## 11           0.9078947        0.8157895
## 12           0.8684211        0.7368421
## 13           0.9210526        0.8421053
## 14           0.9736842        0.9473684
## 15           0.9210526        0.8421053
## 16           0.9736842        0.9473684
## 17           0.9210526        0.8421053
## 18           0.9210526        0.8421053
## 19           1.0000000        1.0000000
## 20           0.9605263        0.9210526
## 21           0.9078947        0.8157895
## 22           0.9210526        0.8421053
## 23           0.9473684        0.8947368
## 24           0.9868421        0.9736842
## 25           1.0000000        1.0000000
## 26           0.9210526        0.8421053
## 27           0.9736842        0.9473684
## 28           0.9473684        0.8947368
## 29           0.9078947        0.8157895
## 30           0.9078947        0.8157895

summary(results)

## 
## Call:
## summary.resamples(object = results)
## 
## Models: Linear, Radial, Polynomial 
## Number of resamples: 30 
## 
## Accuracy 
##                 Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
## Linear     0.7763158 0.8289474 0.8750000 0.8640351 0.9046053 0.9473684    0
## Radial     0.8289474 0.9210526 0.9407895 0.9390351 0.9605263 1.0000000    0
## Polynomial 0.8421053 0.9210526 0.9407895 0.9399123 0.9736842 1.0000000    0
## 
## Kappa 
##                 Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
## Linear     0.5526316 0.6578947 0.7500000 0.7280702 0.8092105 0.8947368    0
## Radial     0.6578947 0.8421053 0.8815789 0.8780702 0.9210526 1.0000000    0
## Polynomial 0.6842105 0.8421053 0.8815789 0.8798246 0.9473684 1.0000000    0

bwplot(results, metric="Accuracy")

Hier nochmal ein Vergleich der Accuracy der verschiedenen Kernel auf die Trainingsdaten. Wie man sieht ist hier der Radial Kernel der präziseste.

modell <- c("SVM Linear", "SVM Radial", "SVM Polynomial", "SVM RadialSigma")
accuracies <- c(CM_linear$overall[1], CM_radial$overall[1], CM_poly$overall[1], CM_RS$overall[1])
sensitivities <- c(CM_linear$byClass[1], CM_radial$byClass[1], CM_poly$byClass[1], CM_RS$byClass[1])
specificities <- c(CM_linear$byClass[2], CM_radial$byClass[2], CM_poly$byClass[2], CM_RS$byClass[2])
results_svm = data.frame(
  "Modell" = modell,
  "Sensitivity" = sensitivities,
  "Specificity" = specificities,
  "Test Accuracy" = accuracies
)

kable_styling(kable(results_svm, format = "html", digits = 4), full_width = FALSE)

Modell	Sensitivity	Specificity	Test.Accuracy
SVM Linear	0.9767	0.4737	0.8857
SVM Radial	0.9773	0.5294	0.9048
SVM Polynomial	0.9655	0.4444	0.8762
SVM RadialSigma	0.9789	0.9000	0.9714

Wie man in der abschließenden Ergbnistabelle erkennen kann, performt das Radial Sigma Modell bzgl. Sensitivity ähnlich wie die anderen Modelle, hat aber eine weit höhere Specivity, was zu einer massiv besseren Accuracy führt. Dadurch ist dieses Modell auf jeden Fall zu Präferieren auf seiten der SVMs

Neuronale Netze

In diesem Abschnitt wird versucht mit Hilfe von verschiedenen Neuronalen Netzen die Vorhersage einer Corona Krankheit zu verbessern.

data_train <- data_clean[train_idx, ]
data_test <- data_clean[-train_idx, ]

ggplot(data=data_train, aes(data_train$target)) + 
  geom_histogram(stat = "count")

## Warning: Ignoring unknown parameters: binwidth, bins, pad

Die Responsevariable im Trainingsdatensatz weist eine ziemlich starke Imbalance aus, es liegt somit das bekannte Rare Class Problem bei der Klassifikation vor.

table(data_train$target)

## 
##   0   1 
## 380  47

Um eine bessere Trainingsgrundlage für das Neuronale Netz zu haben, führen wir ein Upsampling der Trainingsdaten durch. Damit beheben wir das Rare Class Problem im Trainingsdatensatz. Nun haben wir im Trainingsdatensatz jeweils 1062 Corona Infizierte und 1062 Nicht Corona Infizierte.

set.seed(1910837262)
up_train_nn <- upSample(x = data_train[, -ncol(data_train)],
                     y = as.factor(data_train$target))                         
table(up_train_nn$target) 

## 
##   0   1 
## 380 380

up_train_nn <- up_train_nn %>%
    select(-Class)
print(str(up_train_nn))

## 'data.frame':    760 obs. of  16 variables:
##  $ Patient.age.quantile                                 : num  17 1 9 11 13 9 17 17 19 10 ...
##  $ target                                               : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ Patient.addmited.to.regular.ward..1.yes..0.no.       : Factor w/ 2 levels "0","1": 1 1 1 1 1 2 1 1 1 1 ...
##  $ Patient.addmited.to.semi.intensive.unit..1.yes..0.no.: Factor w/ 2 levels "0","1": 1 2 1 1 1 1 1 1 2 1 ...
##  $ Patient.addmited.to.intensive.care.unit..1.yes..0.no.: Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...
##  $ sickness                                             : Factor w/ 2 levels "0","1": 2 1 2 2 2 2 2 2 2 1 ...
##  $ Hematocrit                                           : num  0.237 -1.572 -0.748 0.992 1.015 ...
##  $ Platelets                                            : num  -0.517 1.43 -0.429 0.073 -0.178 ...
##  $ Mean.platelet.volume                                 : num  0.0107 -1.6722 -0.2137 -0.5503 0.796 ...
##  $ Lymphocytes                                          : num  0.31837 -0.00574 -1.11451 0.04544 -0.73071 ...
##  $ Mean.corpuscular.hemoglobin.concentration..MCHC.     : num  -0.951 3.331 0.543 -0.453 -0.353 ...
##  $ Leukocytes                                           : num  -0.0946 0.3646 -0.8849 -0.2115 -0.0751 ...
##  $ Basophils                                            : num  -0.2238 -0.2238 0.0817 -0.8347 2.5254 ...
##  $ Mean.corpuscular.hemoglobin..MCH.                    : num  -0.292 0.178 1.746 0.335 0.544 ...
##  $ Eosinophils                                          : num  1.482 1.019 -0.667 -0.709 0.218 ...
##  $ Monocytes                                            : num  0.3575 0.0687 1.2768 -0.2202 0.0687 ...
## NULL

data_test_x <- data_test %>%
  select(-target)

Bevor wir das Neuronale Netz trainieren müssen wir zu erst die Inputdaten als zu einer Matrix umwandeln.

library(neuralnet)

## 
## Attaching package: 'neuralnet'

## The following object is masked from 'package:dplyr':
## 
##     compute

#preprocessParams <- preProcess(up_train_nn, method=c("scale"))

up_train_nn_matrix <-  as.matrix(sapply(up_train_nn, as.numeric))


modell_nn1 <- neuralnet(target ~., data = up_train_nn_matrix, hidden=c(10), linear.output = FALSE)

Das erste Neuronale Netz wurde mit der Library Neuralnet trainiert. Die Inputdaten wurden dabei nicht skaliert und es wurde alle Features aus dem Trainingsdatensatz verwendet, um die Targetvariable vorherzusagen. Dazu haben wir im ersten Versuch ein Neuronales Netz mit einem Hiddenlayer und 10 Neuronen verwendet.

plot(modell_nn1, rep = "best")

Nun wollen wir die Genauigkeit des Netzes auf den Testdaten errechnen.

data_test_nn_x <- data_test %>%
  select(-target)
data_test_nn_x <-  as.matrix(sapply(data_test_nn_x, as.numeric))

predict_testNN_1 = compute(modell_nn1, data_test_nn_x)
predict_testNN_1<-sapply(predict_testNN_1$net.result,round,digits=0)
nn_table1 <- table(data_test$target, predict_testNN_1)

results_nn1 <- data.frame(actual = data_test$target, prediction = predict_testNN_1)
#attach(results_nn1)
nn_table1

##    predict_testNN_1
##      1
##   0 94
##   1 11

library(nnet)
up_train_nn$target = class.ind(up_train_nn$target)
data_test_nn <- data_test
data_test_nn$target = class.ind(data_test_nn$target) 
data_test_nn_x <- data_test_nn %>%
  select(-target)

modell_nn2 <- nnet(target ~ ., data = up_train_nn, size = 2, rang = 0.1, maxit = 200, decay=5e-4, softmax = TRUE )

## # weights:  38
## initial  value 527.040057 
## iter  10 value 452.372850
## iter  20 value 391.383463
## iter  30 value 321.530598
## iter  40 value 305.369051
## iter  50 value 295.077802
## iter  60 value 293.577716
## iter  70 value 291.497938
## iter  80 value 276.942523
## iter  90 value 251.193745
## iter 100 value 239.385363
## iter 110 value 214.921572
## iter 120 value 206.363383
## iter 130 value 200.540603
## iter 140 value 195.243433
## iter 150 value 184.831926
## iter 160 value 178.861755
## iter 170 value 178.054813
## iter 180 value 177.615947
## iter 190 value 174.815629
## iter 200 value 173.358834
## final  value 173.358834 
## stopped after 200 iterations

#import the function from Github
library(devtools)

## Loading required package: usethis

source_url('https://gist.githubusercontent.com/fawda123/7471137/raw/466c1474d0a505ff044412703516c34f1a4684a5/nnet_plot_update.r')

## SHA-1 hash of file is 74c80bd5ddbc17ab3ae5ece9c0ed9beb612e87ef

plot.nnet(modell_nn2)

## Loading required package: scales

## Loading required package: reshape

## 
## Attaching package: 'reshape'

## The following object is masked from 'package:dplyr':
## 
##     rename

predict_testNN_2 <- predict(modell_nn2, data_test_nn_x)
predict_testNN_2<-sapply(predict_testNN_2,round,digits=0)

#table(data_test_nn$target[,2], predict_testNN_2[107:212])

conf_nn2 <-confusionMatrix(table(data_test_nn$target[,2], predict_testNN_2[105:209]))
conf_nn2

## Confusion Matrix and Statistics
## 
##    
##      0  1
##   0 72 22
##   1  6  5
##                                           
##                Accuracy : 0.7333          
##                  95% CI : (0.6381, 0.8149)
##     No Information Rate : 0.7429          
##     P-Value [Acc > NIR] : 0.637117        
##                                           
##                   Kappa : 0.1343          
##                                           
##  Mcnemar's Test P-Value : 0.004586        
##                                           
##             Sensitivity : 0.9231          
##             Specificity : 0.1852          
##          Pos Pred Value : 0.7660          
##          Neg Pred Value : 0.4545          
##              Prevalence : 0.7429          
##          Detection Rate : 0.6857          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.5541          
##                                           
##        'Positive' Class : 0               
## 

Dieses einfache Neuronale Netz ommt auf eine Accuracy von etwas mehr als 76 %, dies hat auf diesen Daten aber relativ wenig Aussagekraft. Die Specificity ist die Kennzahl, die uns auf unseren Daten am meisten interessiert. Dabei kommt das Modell nur auf 23 %.

acc_nn2 <- conf_nn2$overall[1]
sens_nn2 <- conf_nn2$byClass[1]
spec_nn2 <- conf_nn2$byClass[2]

Skalieren und One Hot Encoden der Daten:

Nun trainieren wir dasselbe Neuronale Netz nur mit vorverarbeiteten Daten, das heißt wir Dummy Encoden die Faktorvariablen und wir normalisieren die numerischen Daten. Danach werden wir prüfen, ob sich dadurch das Modell signifikant verbessern konnte.

glimpse(up_train_nn)

## Rows: 760
## Columns: 16
## $ Patient.age.quantile                                  <dbl> 17, 1, 9, 11, 1…
## $ target                                                <dbl[,2]> <matrix[26 …
## $ Patient.addmited.to.regular.ward..1.yes..0.no.        <fct> 0, 0, 0, 0, 0, …
## $ Patient.addmited.to.semi.intensive.unit..1.yes..0.no. <fct> 0, 1, 0, 0, 0, …
## $ Patient.addmited.to.intensive.care.unit..1.yes..0.no. <fct> 0, 0, 0, 0, 0, …
## $ sickness                                              <fct> 1, 0, 1, 1, 1, …
## $ Hematocrit                                            <dbl> 0.23651545, -1.…
## $ Platelets                                             <dbl> -0.51741302, 1.…
## $ Mean.platelet.volume                                  <dbl> 0.01067657, -1.…
## $ Lymphocytes                                           <dbl> 0.318365753, -0…
## $ Mean.corpuscular.hemoglobin.concentration..MCHC.      <dbl> -0.95079035, 3.…
## $ Leukocytes                                            <dbl> -9.461035e-02, …
## $ Basophils                                             <dbl> -0.22376651, -0…
## $ Mean.corpuscular.hemoglobin..MCH.                     <dbl> -0.29226932, 0.…
## $ Eosinophils                                           <dbl> 1.48215818, 1.0…
## $ Monocytes                                             <dbl> 0.35754666, 0.0…

library(ade4)
library(data.table)

## 
## Attaching package: 'data.table'

## The following object is masked from 'package:reshape':
## 
##     melt

## The following objects are masked from 'package:dplyr':
## 
##     between, first, last

ohe_feats = c( 'Patient.addmited.to.regular.ward..1.yes..0.no.', 'Patient.addmited.to.semi.intensive.unit..1.yes..0.no.', 'Patient.addmited.to.intensive.care.unit..1.yes..0.no.', "sickness")
for (f in ohe_feats){
  df_all_dummy = acm.disjonctif(up_train_nn[f])
  up_train_nn[f] = NULL
  up_train_nn = cbind(up_train_nn, df_all_dummy)
}

ohe_feats = c('Patient.addmited.to.regular.ward..1.yes..0.no.', 'Patient.addmited.to.semi.intensive.unit..1.yes..0.no.', 'Patient.addmited.to.intensive.care.unit..1.yes..0.no.', "sickness")

for (f in ohe_feats){
  df_all_dummy = acm.disjonctif(data_test_nn[f])
  data_test_nn[f] = NULL
  data_test_nn = cbind(data_test_nn, df_all_dummy)
}

preProcValues <- preProcess(up_train_nn, method = c("center", "scale"))

up_train_nn_transformed <- predict(preProcValues, up_train_nn)
data_test_nn_transformed <- predict(preProcValues, data_test_nn)

data_test_nn_transformed_x <- data_test_nn_transformed %>%
  select(-target)

modell_nn3 <- nnet(target ~ ., data = up_train_nn_transformed, size = 2, rang = 0.1, maxit = 200, decay=5e-4, softmax = TRUE )

## # weights:  46
## initial  value 527.163385 
## iter  10 value 334.507677
## iter  20 value 278.597294
## iter  30 value 220.360043
## iter  40 value 198.271403
## iter  50 value 179.789451
## iter  60 value 174.287902
## iter  70 value 173.924387
## iter  80 value 173.679228
## iter  90 value 173.359790
## iter 100 value 173.063926
## iter 110 value 172.927924
## iter 120 value 172.916586
## iter 130 value 172.909744
## iter 140 value 172.880737
## iter 150 value 172.861204
## iter 160 value 172.860771
## iter 170 value 172.860499
## final  value 172.860495 
## converged

plot.nnet(modell_nn3)

predict_testNN_3 <- predict(modell_nn3, data_test_nn_transformed_x)[,2]
predict_testNN_3<-sapply(predict_testNN_3,round,digits=0)

results_nn3 <- data.frame(actual = data_test_nn$target, prediction = predict_testNN_3)
#attach(results_nn3)

table(results_nn3$actual.1,results_nn3$prediction)

##    
##      0  1
##   0 81 13
##   1  3  8

conf_nn3 <- confusionMatrix(table(results_nn3$actual.1,results_nn3$prediction))
conf_nn3

## Confusion Matrix and Statistics
## 
##    
##      0  1
##   0 81 13
##   1  3  8
##                                           
##                Accuracy : 0.8476          
##                  95% CI : (0.7644, 0.9103)
##     No Information Rate : 0.8             
##     P-Value [Acc > NIR] : 0.13470         
##                                           
##                   Kappa : 0.4203          
##                                           
##  Mcnemar's Test P-Value : 0.02445         
##                                           
##             Sensitivity : 0.9643          
##             Specificity : 0.3810          
##          Pos Pred Value : 0.8617          
##          Neg Pred Value : 0.7273          
##              Prevalence : 0.8000          
##          Detection Rate : 0.7714          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.6726          
##                                           
##        'Positive' Class : 0               
## 

Das Modell mit den skalierten und encodeten Inputdaten schneidet doch deutlich besser ab. Die Specificity kommt auf 38%, was zwar immer noch nicht gut ist, aber immerhin schon einmal eine deutliche Verbesserung zum vorherigen Modell. Dieses Modell hat 3 Patienten fälschlicherweise als gesund ausgegeben, obwohl der Patient mit Corona infiziert ist.

acc_nn3 <- conf_nn3$overall[1]
sens_nn3 <- conf_nn3$byClass[1]
spec_nn3 <- conf_nn3$byClass[2]

Nun trainieren wir noch ein Neuronale Netz Modell mit der Library Caret. Dazu werden wir die Inputdaten auch preprocessen und eine 10 Fold Cross Validation anwenden, die wir 3 mal wiederholen. Wir verwenden hier erstmal den “normalen” Trainingsdatensatz und nicht den upgesampleten.

#Caret Modell
TrainingParameters_nn <- trainControl(method = "repeatedcv", number = 10, repeats=3)
modell_nn4 <- train(data_train[,-2], data_train$target,
                  method = "nnet",
                  trControl= TrainingParameters_nn,
                  preProcess=c("scale","center"),
                  na.action = na.omit
)

## # weights:  19
## initial  value 185.408160 
## iter  10 value 84.290613
## iter  20 value 66.778235
## iter  30 value 61.399709
## iter  40 value 59.552666
## iter  50 value 55.115770
## final  value 54.992022 
## converged
## # weights:  55
## initial  value 190.705179 
## iter  10 value 55.844046
## iter  20 value 45.070703
## iter  30 value 42.208468
## iter  40 value 38.894742
## iter  50 value 37.847652
## iter  60 value 36.956438
## iter  70 value 36.939003
## iter  80 value 36.935532
## iter  90 value 36.935020
## final  value 36.935018 
## converged
## # weights:  91
## initial  value 330.173709 
## iter  10 value 61.490610
## iter  20 value 38.691226
## iter  30 value 22.728135
## iter  40 value 17.656707
## iter  50 value 16.299636
## iter  60 value 15.758677
## iter  70 value 15.597162
## iter  80 value 15.490489
## iter  90 value 15.401558
## iter 100 value 15.319556
## final  value 15.319556 
## stopped after 100 iterations
## # weights:  19
## initial  value 222.405721 
## iter  10 value 74.314836
## iter  20 value 67.279782
## iter  30 value 66.752609
## final  value 66.649516 
## converged
## # weights:  55
## initial  value 241.343168 
## iter  10 value 74.032693
## iter  20 value 62.448293
## iter  30 value 57.093623
## iter  40 value 56.339582
## iter  50 value 55.786909
## iter  60 value 55.417339
## iter  70 value 55.320527
## iter  80 value 55.314192
## final  value 55.314191 
## converged
## # weights:  91
## initial  value 286.627491 
## iter  10 value 66.202031
## iter  20 value 55.115106
## iter  30 value 51.251889
## iter  40 value 49.250593
## iter  50 value 48.372418
## iter  60 value 48.118049
## iter  70 value 48.077961
## iter  80 value 48.065241
## iter  90 value 48.063338
## final  value 48.063326 
## converged
## # weights:  19
## initial  value 191.540826 
## iter  10 value 73.093334
## iter  20 value 57.104328
## iter  30 value 51.552355
## iter  40 value 50.282335
## iter  50 value 50.210799
## iter  60 value 50.173916
## iter  70 value 50.166243
## iter  80 value 50.162650
## iter  90 value 50.161904
## iter 100 value 50.160884
## final  value 50.160884 
## stopped after 100 iterations
## # weights:  55
## initial  value 382.035585 
## iter  10 value 61.164785
## iter  20 value 47.019382
## iter  30 value 38.227861
## iter  40 value 33.552050
## iter  50 value 26.059658
## iter  60 value 25.128027
## iter  70 value 24.335718
## iter  80 value 24.190858
## iter  90 value 24.093928
## iter 100 value 24.002794
## final  value 24.002794 
## stopped after 100 iterations
## # weights:  91
## initial  value 206.683352 
## iter  10 value 62.308485
## iter  20 value 34.731843
## iter  30 value 18.154430
## iter  40 value 13.683318
## iter  50 value 13.222837
## iter  60 value 13.044795
## iter  70 value 12.956077
## iter  80 value 12.870555
## iter  90 value 12.804837
## iter 100 value 12.746638
## final  value 12.746638 
## stopped after 100 iterations
## # weights:  19
## initial  value 286.277621 
## iter  10 value 89.527597
## iter  20 value 70.726338
## iter  30 value 63.210512
## iter  40 value 56.224796
## iter  50 value 55.692410
## iter  60 value 55.228112
## iter  70 value 54.228625
## iter  80 value 54.224239
## iter  90 value 54.221442
## iter 100 value 54.220771
## final  value 54.220771 
## stopped after 100 iterations
## # weights:  55
## initial  value 224.875887 
## iter  10 value 48.945961
## iter  20 value 29.915020
## iter  30 value 24.959051
## iter  40 value 23.431417
## iter  50 value 23.070004
## iter  60 value 23.058013
## iter  70 value 23.057749
## final  value 23.057614 
## converged
## # weights:  91
## initial  value 239.797675 
## iter  10 value 73.742683
## iter  20 value 35.225347
## iter  30 value 24.137757
## iter  40 value 22.360214
## iter  50 value 21.796791
## iter  60 value 20.955173
## iter  70 value 19.240365
## iter  80 value 17.411884
## iter  90 value 16.403695
## iter 100 value 15.727525
## final  value 15.727525 
## stopped after 100 iterations
## # weights:  19
## initial  value 282.533332 
## iter  10 value 82.056542
## iter  20 value 64.084308
## iter  30 value 62.165219
## final  value 62.032149 
## converged
## # weights:  55
## initial  value 198.200463 
## iter  10 value 67.293302
## iter  20 value 58.699726
## iter  30 value 53.747390
## iter  40 value 51.819863
## iter  50 value 51.331432
## iter  60 value 51.283131
## iter  70 value 51.274482
## final  value 51.274459 
## converged
## # weights:  91
## initial  value 205.820665 
## iter  10 value 64.517671
## iter  20 value 51.252143
## iter  30 value 49.971449
## iter  40 value 49.631921
## iter  50 value 49.339827
## iter  60 value 48.196932
## iter  70 value 47.492840
## iter  80 value 47.202777
## iter  90 value 46.990391
## iter 100 value 46.865842
## final  value 46.865842 
## stopped after 100 iterations
## # weights:  19
## initial  value 183.074496 
## iter  10 value 67.391766
## iter  20 value 51.370412
## iter  30 value 46.108455
## iter  40 value 45.294900
## iter  50 value 45.271980
## iter  60 value 45.248163
## iter  70 value 45.243476
## iter  80 value 45.242443
## iter  90 value 45.242072
## iter 100 value 45.241955
## final  value 45.241955 
## stopped after 100 iterations
## # weights:  55
## initial  value 203.435995 
## iter  10 value 46.087275
## iter  20 value 33.179598
## iter  30 value 28.985725
## iter  40 value 28.506676
## iter  50 value 28.385981
## iter  60 value 27.490563
## iter  70 value 27.156316
## iter  80 value 27.075757
## iter  90 value 27.024519
## iter 100 value 26.979727
## final  value 26.979727 
## stopped after 100 iterations
## # weights:  91
## initial  value 220.032032 
## iter  10 value 52.488586
## iter  20 value 29.894010
## iter  30 value 19.538080
## iter  40 value 16.035090
## iter  50 value 15.076600
## iter  60 value 14.668283
## iter  70 value 14.180792
## iter  80 value 13.854090
## iter  90 value 13.702398
## iter 100 value 13.585733
## final  value 13.585733 
## stopped after 100 iterations
## # weights:  19
## initial  value 311.648093 
## iter  10 value 73.007434
## iter  20 value 63.024575
## iter  30 value 61.189028
## iter  40 value 59.559704
## iter  50 value 58.924662
## iter  60 value 57.782550
## iter  70 value 55.312979
## iter  80 value 55.145477
## iter  90 value 55.106892
## iter 100 value 55.090757
## final  value 55.090757 
## stopped after 100 iterations
## # weights:  55
## initial  value 384.137809 
## iter  10 value 63.771419
## iter  20 value 45.285093
## iter  30 value 36.843221
## iter  40 value 32.704915
## iter  50 value 31.122020
## iter  60 value 30.346372
## iter  70 value 29.335483
## iter  80 value 28.419185
## iter  90 value 27.667624
## iter 100 value 27.552865
## final  value 27.552865 
## stopped after 100 iterations
## # weights:  91
## initial  value 330.109239 
## iter  10 value 62.101787
## iter  20 value 37.195417
## iter  30 value 32.848913
## iter  40 value 30.607429
## iter  50 value 29.447729
## iter  60 value 28.374704
## iter  70 value 27.374718
## iter  80 value 25.870598
## iter  90 value 25.743837
## iter 100 value 25.716873
## final  value 25.716873 
## stopped after 100 iterations
## # weights:  19
## initial  value 306.238629 
## iter  10 value 81.693083
## iter  20 value 75.505622
## iter  30 value 70.577848
## iter  40 value 66.814156
## iter  50 value 66.790546
## final  value 66.789381 
## converged
## # weights:  55
## initial  value 204.817264 
## iter  10 value 73.928894
## iter  20 value 63.163005
## iter  30 value 55.972889
## iter  40 value 54.698212
## iter  50 value 54.178890
## iter  60 value 53.938935
## iter  70 value 53.085333
## iter  80 value 52.608627
## iter  90 value 52.562065
## final  value 52.561919 
## converged
## # weights:  91
## initial  value 430.976164 
## iter  10 value 68.605146
## iter  20 value 57.334478
## iter  30 value 53.611358
## iter  40 value 52.238105
## iter  50 value 51.513395
## iter  60 value 50.805763
## iter  70 value 50.574978
## iter  80 value 50.470421
## iter  90 value 50.359608
## iter 100 value 50.350961
## final  value 50.350961 
## stopped after 100 iterations
## # weights:  19
## initial  value 232.768030 
## iter  10 value 83.425363
## iter  20 value 61.401918
## iter  30 value 59.195269
## iter  40 value 58.120347
## iter  50 value 58.017405
## iter  60 value 57.972578
## iter  70 value 57.959407
## iter  80 value 57.958188
## iter  90 value 57.957476
## iter 100 value 57.956749
## final  value 57.956749 
## stopped after 100 iterations
## # weights:  55
## initial  value 349.967968 
## iter  10 value 71.072014
## iter  20 value 46.818952
## iter  30 value 37.009311
## iter  40 value 32.394619
## iter  50 value 31.447583
## iter  60 value 30.654918
## iter  70 value 30.194055
## iter  80 value 29.721778
## iter  90 value 29.490688
## iter 100 value 29.412036
## final  value 29.412036 
## stopped after 100 iterations
## # weights:  91
## initial  value 182.815797 
## iter  10 value 63.954774
## iter  20 value 42.763751
## iter  30 value 38.271554
## iter  40 value 36.647291
## iter  50 value 34.901824
## iter  60 value 34.440258
## iter  70 value 34.146933
## iter  80 value 33.853376
## iter  90 value 33.660804
## iter 100 value 33.498307
## final  value 33.498307 
## stopped after 100 iterations
## # weights:  19
## initial  value 387.605790 
## iter  10 value 69.913322
## iter  20 value 54.545131
## iter  30 value 48.482722
## iter  40 value 46.644491
## iter  50 value 45.872750
## iter  60 value 45.652011
## iter  70 value 44.629535
## iter  80 value 43.421931
## iter  90 value 43.294435
## iter 100 value 43.150734
## final  value 43.150734 
## stopped after 100 iterations
## # weights:  55
## initial  value 579.606959 
## iter  10 value 56.457016
## iter  20 value 42.327936
## iter  30 value 38.169519
## iter  40 value 35.910577
## iter  50 value 35.530869
## iter  60 value 35.519710
## iter  70 value 35.519379
## final  value 35.519378 
## converged
## # weights:  91
## initial  value 537.179777 
## iter  10 value 47.722478
## iter  20 value 24.277198
## iter  30 value 16.904076
## iter  40 value 15.021183
## iter  50 value 14.585503
## iter  60 value 14.291731
## iter  70 value 14.234319
## iter  80 value 14.227045
## iter  90 value 14.186040
## iter 100 value 14.183905
## final  value 14.183905 
## stopped after 100 iterations
## # weights:  19
## initial  value 303.157754 
## iter  10 value 74.107947
## iter  20 value 59.409695
## iter  30 value 57.125317
## iter  40 value 57.076137
## final  value 57.076136 
## converged
## # weights:  55
## initial  value 330.815333 
## iter  10 value 70.756526
## iter  20 value 56.840547
## iter  30 value 51.877600
## iter  40 value 49.005727
## iter  50 value 48.258565
## iter  60 value 48.212065
## iter  70 value 48.206692
## final  value 48.206599 
## converged
## # weights:  91
## initial  value 166.137564 
## iter  10 value 55.267976
## iter  20 value 47.329214
## iter  30 value 42.350944
## iter  40 value 41.518976
## iter  50 value 41.347030
## iter  60 value 41.307753
## iter  70 value 41.264149
## iter  80 value 41.256458
## final  value 41.256454 
## converged
## # weights:  19
## initial  value 354.503141 
## iter  10 value 70.777513
## iter  20 value 54.093499
## iter  30 value 48.309698
## iter  40 value 47.419172
## iter  50 value 47.155313
## iter  60 value 46.782899
## iter  70 value 45.975276
## iter  80 value 45.718713
## iter  90 value 45.682438
## iter 100 value 45.664932
## final  value 45.664932 
## stopped after 100 iterations
## # weights:  55
## initial  value 270.436308 
## iter  10 value 56.197950
## iter  20 value 40.853311
## iter  30 value 35.808824
## iter  40 value 30.318804
## iter  50 value 28.043337
## iter  60 value 27.403397
## iter  70 value 27.315056
## iter  80 value 27.281087
## iter  90 value 27.253650
## iter 100 value 27.223381
## final  value 27.223381 
## stopped after 100 iterations
## # weights:  91
## initial  value 549.781329 
## iter  10 value 56.572117
## iter  20 value 22.968796
## iter  30 value 14.973051
## iter  40 value 9.394731
## iter  50 value 8.818811
## iter  60 value 8.654159
## iter  70 value 8.530771
## iter  80 value 8.449709
## iter  90 value 8.384559
## iter 100 value 8.331458
## final  value 8.331458 
## stopped after 100 iterations
## # weights:  19
## initial  value 345.040686 
## iter  10 value 75.919085
## iter  20 value 66.020282
## iter  30 value 60.273975
## iter  40 value 57.654942
## iter  50 value 55.763388
## iter  60 value 50.449853
## iter  70 value 48.026831
## iter  80 value 47.149169
## iter  90 value 47.130671
## iter 100 value 47.125554
## final  value 47.125554 
## stopped after 100 iterations
## # weights:  55
## initial  value 374.754083 
## iter  10 value 62.482973
## iter  20 value 44.339400
## iter  30 value 39.607140
## iter  40 value 34.729597
## iter  50 value 32.907114
## iter  60 value 30.425710
## iter  70 value 29.617660
## iter  80 value 29.150846
## iter  90 value 28.835879
## iter 100 value 28.664701
## final  value 28.664701 
## stopped after 100 iterations
## # weights:  91
## initial  value 306.525480 
## iter  10 value 71.309038
## iter  20 value 40.591867
## iter  30 value 31.650674
## iter  40 value 30.155208
## iter  50 value 28.738762
## iter  60 value 27.145220
## iter  70 value 24.747099
## iter  80 value 24.367005
## iter  90 value 23.428498
## iter 100 value 22.365982
## final  value 22.365982 
## stopped after 100 iterations
## # weights:  19
## initial  value 316.023728 
## iter  10 value 83.252680
## iter  20 value 67.945097
## iter  30 value 65.046944
## iter  40 value 64.926625
## iter  50 value 64.925987
## final  value 64.925961 
## converged
## # weights:  55
## initial  value 289.372860 
## iter  10 value 68.942698
## iter  20 value 58.598883
## iter  30 value 57.514146
## iter  40 value 56.556629
## iter  50 value 56.478624
## iter  60 value 56.477083
## final  value 56.477016 
## converged
## # weights:  91
## initial  value 448.146538 
## iter  10 value 68.092229
## iter  20 value 53.970296
## iter  30 value 50.009618
## iter  40 value 48.393135
## iter  50 value 48.154222
## iter  60 value 47.797771
## iter  70 value 47.744097
## iter  80 value 47.723362
## iter  90 value 47.715403
## final  value 47.715392 
## converged
## # weights:  19
## initial  value 243.053167 
## iter  10 value 76.654603
## iter  20 value 61.727120
## iter  30 value 60.079793
## iter  40 value 56.975127
## iter  50 value 55.121277
## iter  60 value 50.192606
## iter  70 value 48.534593
## iter  80 value 48.442359
## iter  90 value 48.411481
## iter 100 value 48.405476
## final  value 48.405476 
## stopped after 100 iterations
## # weights:  55
## initial  value 317.214494 
## iter  10 value 55.063601
## iter  20 value 41.137647
## iter  30 value 35.922664
## iter  40 value 34.102635
## iter  50 value 33.426190
## iter  60 value 33.285206
## iter  70 value 33.223503
## iter  80 value 33.181476
## iter  90 value 33.135805
## iter 100 value 33.109084
## final  value 33.109084 
## stopped after 100 iterations
## # weights:  91
## initial  value 261.807770 
## iter  10 value 51.401464
## iter  20 value 32.930436
## iter  30 value 22.808936
## iter  40 value 21.816832
## iter  50 value 21.314356
## iter  60 value 20.598306
## iter  70 value 18.921026
## iter  80 value 18.522722
## iter  90 value 18.255729
## iter 100 value 18.102885
## final  value 18.102885 
## stopped after 100 iterations
## # weights:  19
## initial  value 251.247557 
## iter  10 value 77.695291
## iter  20 value 61.167949
## iter  30 value 59.988937
## iter  40 value 56.825569
## iter  50 value 56.613582
## iter  60 value 56.569331
## iter  70 value 56.551488
## iter  80 value 56.531075
## iter  90 value 56.523204
## iter 100 value 56.516267
## final  value 56.516267 
## stopped after 100 iterations
## # weights:  55
## initial  value 203.579865 
## iter  10 value 57.996361
## iter  20 value 45.832417
## iter  30 value 41.337327
## iter  40 value 37.886366
## iter  50 value 36.263010
## iter  60 value 35.445199
## iter  70 value 35.358661
## iter  80 value 35.333661
## iter  90 value 35.288182
## iter 100 value 35.266811
## final  value 35.266811 
## stopped after 100 iterations
## # weights:  91
## initial  value 572.618555 
## iter  10 value 67.542340
## iter  20 value 48.388935
## iter  30 value 35.916410
## iter  40 value 29.574800
## iter  50 value 23.765106
## iter  60 value 21.199714
## iter  70 value 20.354820
## iter  80 value 20.101955
## iter  90 value 19.781746
## iter 100 value 19.008977
## final  value 19.008977 
## stopped after 100 iterations
## # weights:  19
## initial  value 342.621679 
## iter  10 value 69.260450
## iter  20 value 65.220521
## iter  30 value 65.186753
## final  value 65.186732 
## converged
## # weights:  55
## initial  value 165.757442 
## iter  10 value 74.292964
## iter  20 value 62.600371
## iter  30 value 56.797474
## iter  40 value 54.066879
## iter  50 value 53.508331
## iter  60 value 52.617546
## iter  70 value 50.747981
## iter  80 value 50.503616
## iter  90 value 50.499290
## final  value 50.499241 
## converged
## # weights:  91
## initial  value 432.666688 
## iter  10 value 76.598614
## iter  20 value 57.579174
## iter  30 value 52.727326
## iter  40 value 51.357709
## iter  50 value 50.024186
## iter  60 value 49.344433
## iter  70 value 49.091518
## iter  80 value 48.583675
## iter  90 value 48.069694
## iter 100 value 47.596017
## final  value 47.596017 
## stopped after 100 iterations
## # weights:  19
## initial  value 407.422633 
## iter  10 value 115.434875
## iter  20 value 91.229537
## iter  30 value 70.535249
## iter  40 value 64.348726
## iter  50 value 64.194124
## iter  60 value 63.595239
## iter  70 value 58.377638
## iter  80 value 56.836354
## iter  90 value 56.664404
## iter 100 value 56.663503
## final  value 56.663503 
## stopped after 100 iterations
## # weights:  55
## initial  value 145.466806 
## iter  10 value 58.818418
## iter  20 value 45.081669
## iter  30 value 38.049279
## iter  40 value 34.466625
## iter  50 value 33.548075
## iter  60 value 32.394313
## iter  70 value 29.240483
## iter  80 value 27.945885
## iter  90 value 27.700734
## iter 100 value 27.587855
## final  value 27.587855 
## stopped after 100 iterations
## # weights:  91
## initial  value 209.486057 
## iter  10 value 54.161432
## iter  20 value 39.264299
## iter  30 value 30.246482
## iter  40 value 28.575760
## iter  50 value 26.639428
## iter  60 value 23.034610
## iter  70 value 21.364055
## iter  80 value 20.333620
## iter  90 value 19.829913
## iter 100 value 19.581526
## final  value 19.581526 
## stopped after 100 iterations
## # weights:  19
## initial  value 218.240339 
## iter  10 value 90.012095
## iter  20 value 56.979914
## iter  30 value 49.936394
## iter  40 value 48.266276
## iter  50 value 47.684868
## iter  60 value 46.357085
## iter  70 value 41.386691
## iter  80 value 40.836878
## iter  90 value 40.836245
## iter 100 value 40.836090
## final  value 40.836090 
## stopped after 100 iterations
## # weights:  55
## initial  value 334.730064 
## iter  10 value 63.644153
## iter  20 value 40.873464
## iter  30 value 37.925274
## iter  40 value 37.150599
## iter  50 value 36.508759
## iter  60 value 35.885653
## iter  70 value 35.707969
## iter  80 value 35.507616
## iter  90 value 35.472238
## iter 100 value 35.432061
## final  value 35.432061 
## stopped after 100 iterations
## # weights:  91
## initial  value 267.436193 
## iter  10 value 79.099267
## iter  20 value 44.920796
## iter  30 value 32.130563
## iter  40 value 22.460950
## iter  50 value 19.259770
## iter  60 value 17.499333
## iter  70 value 16.661115
## iter  80 value 16.240835
## iter  90 value 15.803699
## iter 100 value 15.762513
## final  value 15.762513 
## stopped after 100 iterations
## # weights:  19
## initial  value 222.464673 
## iter  10 value 79.419118
## iter  20 value 64.062096
## iter  30 value 59.529674
## iter  40 value 59.083184
## iter  50 value 59.082664
## final  value 59.082633 
## converged
## # weights:  55
## initial  value 380.353183 
## iter  10 value 71.077013
## iter  20 value 54.850772
## iter  30 value 50.207083
## iter  40 value 49.406652
## iter  50 value 48.995026
## iter  60 value 48.869673
## iter  70 value 48.853687
## final  value 48.853636 
## converged
## # weights:  91
## initial  value 378.824836 
## iter  10 value 62.093280
## iter  20 value 50.364955
## iter  30 value 47.543600
## iter  40 value 47.020366
## iter  50 value 46.340365
## iter  60 value 45.803319
## iter  70 value 45.710542
## iter  80 value 45.489212
## iter  90 value 45.451910
## iter 100 value 45.445913
## final  value 45.445913 
## stopped after 100 iterations
## # weights:  19
## initial  value 396.020545 
## iter  10 value 122.762213
## iter  20 value 112.663964
## iter  30 value 104.384706
## iter  40 value 101.913275
## iter  50 value 97.443649
## iter  60 value 96.733547
## iter  70 value 96.160029
## iter  80 value 96.117528
## iter  90 value 96.097230
## iter 100 value 96.089834
## final  value 96.089834 
## stopped after 100 iterations
## # weights:  55
## initial  value 288.584718 
## iter  10 value 43.631277
## iter  20 value 29.302224
## iter  30 value 26.259462
## iter  40 value 25.416817
## iter  50 value 24.456399
## iter  60 value 24.097394
## iter  70 value 23.826870
## iter  80 value 23.690644
## iter  90 value 23.543142
## iter 100 value 23.431808
## final  value 23.431808 
## stopped after 100 iterations
## # weights:  91
## initial  value 439.276550 
## iter  10 value 54.110665
## iter  20 value 26.332024
## iter  30 value 22.773461
## iter  40 value 22.081642
## iter  50 value 21.980790
## iter  60 value 21.096802
## iter  70 value 20.941940
## iter  80 value 20.672042
## iter  90 value 20.438691
## iter 100 value 20.189340
## final  value 20.189340 
## stopped after 100 iterations
## # weights:  19
## initial  value 168.343722 
## iter  10 value 86.191820
## iter  20 value 72.433935
## iter  30 value 66.571315
## iter  40 value 65.840724
## iter  50 value 64.377752
## iter  60 value 63.362230
## iter  70 value 63.333758
## final  value 63.333082 
## converged
## # weights:  55
## initial  value 261.115398 
## iter  10 value 48.537409
## iter  20 value 31.715875
## iter  30 value 22.539976
## iter  40 value 19.812343
## iter  50 value 18.108177
## iter  60 value 17.166374
## iter  70 value 16.795611
## iter  80 value 16.478567
## iter  90 value 16.184464
## iter 100 value 16.038192
## final  value 16.038192 
## stopped after 100 iterations
## # weights:  91
## initial  value 145.504672 
## iter  10 value 58.771264
## iter  20 value 38.204477
## iter  30 value 31.073672
## iter  40 value 25.055679
## iter  50 value 23.423480
## iter  60 value 22.502783
## iter  70 value 21.071683
## iter  80 value 19.931631
## iter  90 value 18.646069
## iter 100 value 17.690261
## final  value 17.690261 
## stopped after 100 iterations
## # weights:  19
## initial  value 232.894286 
## iter  10 value 76.706554
## iter  20 value 63.325103
## iter  30 value 61.460308
## iter  40 value 60.980898
## final  value 60.980873 
## converged
## # weights:  55
## initial  value 214.229567 
## iter  10 value 60.623066
## iter  20 value 52.465614
## iter  30 value 48.521159
## iter  40 value 47.823761
## iter  50 value 47.359808
## iter  60 value 46.810301
## iter  70 value 45.575723
## iter  80 value 44.865415
## iter  90 value 44.780282
## iter 100 value 44.776703
## final  value 44.776703 
## stopped after 100 iterations
## # weights:  91
## initial  value 356.064199 
## iter  10 value 61.863453
## iter  20 value 47.107724
## iter  30 value 43.803595
## iter  40 value 42.544600
## iter  50 value 42.116174
## iter  60 value 41.887113
## iter  70 value 41.866120
## iter  80 value 41.865336
## iter  90 value 41.865189
## final  value 41.865174 
## converged
## # weights:  19
## initial  value 213.347118 
## iter  10 value 65.167323
## iter  20 value 55.086731
## iter  30 value 51.614278
## iter  40 value 50.364957
## iter  50 value 46.198577
## iter  60 value 45.854288
## iter  70 value 45.778379
## iter  80 value 45.745392
## iter  90 value 45.726474
## iter 100 value 45.724829
## final  value 45.724829 
## stopped after 100 iterations
## # weights:  55
## initial  value 329.215988 
## iter  10 value 57.038067
## iter  20 value 43.392284
## iter  30 value 35.063588
## iter  40 value 24.963449
## iter  50 value 23.233849
## iter  60 value 21.139436
## iter  70 value 20.665133
## iter  80 value 20.561580
## iter  90 value 20.481208
## iter 100 value 20.105253
## final  value 20.105253 
## stopped after 100 iterations
## # weights:  91
## initial  value 367.284884 
## iter  10 value 48.819424
## iter  20 value 25.166973
## iter  30 value 17.776750
## iter  40 value 12.924968
## iter  50 value 11.763557
## iter  60 value 11.508601
## iter  70 value 11.280959
## iter  80 value 11.074781
## iter  90 value 10.939780
## iter 100 value 10.856223
## final  value 10.856223 
## stopped after 100 iterations
## # weights:  19
## initial  value 259.336593 
## iter  10 value 89.993594
## iter  20 value 59.966595
## iter  30 value 56.468876
## iter  40 value 55.095338
## iter  50 value 54.720630
## iter  60 value 54.671383
## iter  70 value 54.651243
## iter  80 value 54.598438
## iter  90 value 54.537963
## iter 100 value 54.510176
## final  value 54.510176 
## stopped after 100 iterations
## # weights:  55
## initial  value 249.423344 
## iter  10 value 58.437503
## iter  20 value 37.349825
## iter  30 value 28.003306
## iter  40 value 26.164021
## iter  50 value 25.770696
## iter  60 value 25.441260
## iter  70 value 25.418247
## iter  80 value 25.415719
## iter  90 value 25.415102
## iter 100 value 25.414853
## final  value 25.414853 
## stopped after 100 iterations
## # weights:  91
## initial  value 313.849454 
## iter  10 value 44.773693
## iter  20 value 22.543577
## iter  30 value 14.970099
## iter  40 value 12.147412
## iter  50 value 11.195229
## iter  60 value 11.103868
## iter  70 value 11.004832
## iter  80 value 10.997777
## iter  90 value 10.972458
## iter 100 value 10.971293
## final  value 10.971293 
## stopped after 100 iterations
## # weights:  19
## initial  value 264.071300 
## iter  10 value 80.045625
## iter  20 value 64.615234
## iter  30 value 63.061080
## iter  40 value 63.019783
## iter  50 value 63.019739
## iter  50 value 63.019738
## final  value 63.019737 
## converged
## # weights:  55
## initial  value 307.415364 
## iter  10 value 69.566024
## iter  20 value 58.269621
## iter  30 value 53.381802
## iter  40 value 52.436233
## iter  50 value 52.182827
## iter  60 value 52.070377
## iter  70 value 52.047902
## final  value 52.047681 
## converged
## # weights:  91
## initial  value 517.461725 
## iter  10 value 76.731564
## iter  20 value 57.799064
## iter  30 value 50.823176
## iter  40 value 48.250555
## iter  50 value 46.289620
## iter  60 value 46.023501
## iter  70 value 45.879392
## iter  80 value 45.841585
## iter  90 value 45.838706
## final  value 45.838689 
## converged
## # weights:  19
## initial  value 237.710549 
## iter  10 value 82.892387
## iter  20 value 61.029888
## iter  30 value 57.711586
## iter  40 value 56.342969
## iter  50 value 53.627385
## iter  60 value 53.500798
## iter  70 value 53.491731
## iter  80 value 53.359836
## iter  90 value 53.271820
## iter 100 value 53.267433
## final  value 53.267433 
## stopped after 100 iterations
## # weights:  55
## initial  value 335.440764 
## iter  10 value 52.370545
## iter  20 value 34.251138
## iter  30 value 28.471382
## iter  40 value 27.970566
## iter  50 value 26.904853
## iter  60 value 25.687843
## iter  70 value 25.162390
## iter  80 value 25.041898
## iter  90 value 24.768853
## iter 100 value 24.683696
## final  value 24.683696 
## stopped after 100 iterations
## # weights:  91
## initial  value 263.637046 
## iter  10 value 55.936004
## iter  20 value 32.952895
## iter  30 value 21.802571
## iter  40 value 18.708880
## iter  50 value 17.512869
## iter  60 value 17.226167
## iter  70 value 17.075609
## iter  80 value 16.980160
## iter  90 value 16.902767
## iter 100 value 16.437062
## final  value 16.437062 
## stopped after 100 iterations
## # weights:  19
## initial  value 309.990241 
## iter  10 value 104.193224
## iter  20 value 78.965869
## iter  30 value 72.698058
## iter  40 value 69.076222
## iter  50 value 69.016690
## iter  60 value 68.988122
## iter  70 value 68.981791
## iter  80 value 68.976439
## iter  90 value 68.972807
## iter 100 value 68.968851
## final  value 68.968851 
## stopped after 100 iterations
## # weights:  55
## initial  value 339.924016 
## iter  10 value 63.079124
## iter  20 value 50.490019
## iter  30 value 48.391569
## iter  40 value 47.851349
## iter  50 value 46.004926
## iter  60 value 44.364665
## iter  70 value 42.950680
## iter  80 value 41.005478
## iter  90 value 40.322223
## iter 100 value 39.876888
## final  value 39.876888 
## stopped after 100 iterations
## # weights:  91
## initial  value 170.199924 
## iter  10 value 50.876607
## iter  20 value 27.657978
## iter  30 value 23.557524
## iter  40 value 21.173226
## iter  50 value 15.599627
## iter  60 value 15.085160
## iter  70 value 14.998271
## iter  80 value 14.987114
## iter  90 value 14.985827
## iter 100 value 14.985430
## final  value 14.985430 
## stopped after 100 iterations
## # weights:  19
## initial  value 362.821151 
## iter  10 value 73.365288
## iter  20 value 62.682115
## iter  30 value 60.950347
## final  value 60.683703 
## converged
## # weights:  55
## initial  value 227.986754 
## iter  10 value 62.373513
## iter  20 value 55.346887
## iter  30 value 52.503163
## iter  40 value 52.021338
## iter  50 value 51.704675
## iter  60 value 51.596328
## iter  70 value 51.593144
## final  value 51.593133 
## converged
## # weights:  91
## initial  value 354.674027 
## iter  10 value 66.790678
## iter  20 value 51.194326
## iter  30 value 48.508613
## iter  40 value 46.196237
## iter  50 value 45.312217
## iter  60 value 45.098840
## iter  70 value 45.027278
## iter  80 value 45.003864
## iter  90 value 45.002977
## final  value 45.002954 
## converged
## # weights:  19
## initial  value 194.917489 
## iter  10 value 68.990723
## iter  20 value 61.359162
## iter  30 value 54.976531
## iter  40 value 53.420243
## iter  50 value 53.271813
## iter  60 value 52.775717
## iter  70 value 52.388151
## iter  80 value 52.242768
## iter  90 value 52.225419
## iter 100 value 52.224259
## final  value 52.224259 
## stopped after 100 iterations
## # weights:  55
## initial  value 236.567182 
## iter  10 value 62.195800
## iter  20 value 45.427642
## iter  30 value 42.080617
## iter  40 value 40.318934
## iter  50 value 39.790329
## iter  60 value 39.746717
## iter  70 value 39.705940
## iter  80 value 39.572616
## iter  90 value 39.456814
## iter 100 value 39.421245
## final  value 39.421245 
## stopped after 100 iterations
## # weights:  91
## initial  value 174.723180 
## iter  10 value 49.715253
## iter  20 value 34.836065
## iter  30 value 19.878326
## iter  40 value 18.398738
## iter  50 value 16.897016
## iter  60 value 16.160773
## iter  70 value 14.569024
## iter  80 value 13.885074
## iter  90 value 13.576471
## iter 100 value 13.274576
## final  value 13.274576 
## stopped after 100 iterations
## # weights:  19
## initial  value 252.090136 
## iter  10 value 82.280792
## iter  20 value 66.825436
## iter  30 value 58.114790
## iter  40 value 56.763178
## iter  50 value 55.847320
## iter  60 value 55.588206
## iter  70 value 54.216535
## iter  80 value 53.574667
## iter  90 value 53.542540
## iter 100 value 53.541960
## final  value 53.541960 
## stopped after 100 iterations
## # weights:  55
## initial  value 182.366944 
## iter  10 value 62.633743
## iter  20 value 47.973995
## iter  30 value 41.718986
## iter  40 value 38.285817
## iter  50 value 37.612332
## iter  60 value 36.591622
## iter  70 value 35.489577
## iter  80 value 34.252665
## iter  90 value 34.223836
## iter 100 value 34.215052
## final  value 34.215052 
## stopped after 100 iterations
## # weights:  91
## initial  value 462.808300 
## iter  10 value 52.058852
## iter  20 value 30.849162
## iter  30 value 24.358450
## iter  40 value 22.216961
## iter  50 value 21.353490
## iter  60 value 21.012662
## iter  70 value 20.979145
## iter  80 value 20.971660
## iter  90 value 20.959464
## iter 100 value 20.950846
## final  value 20.950846 
## stopped after 100 iterations
## # weights:  19
## initial  value 372.316471 
## iter  10 value 73.211804
## iter  20 value 63.315067
## iter  30 value 62.442832
## final  value 62.421381 
## converged
## # weights:  55
## initial  value 283.613796 
## iter  10 value 64.551751
## iter  20 value 54.143111
## iter  30 value 52.108327
## iter  40 value 51.897794
## iter  50 value 51.855283
## iter  60 value 51.842803
## final  value 51.842755 
## converged
## # weights:  91
## initial  value 334.609157 
## iter  10 value 67.093976
## iter  20 value 51.786965
## iter  30 value 47.902467
## iter  40 value 47.151411
## iter  50 value 46.680702
## iter  60 value 46.484701
## iter  70 value 46.372326
## iter  80 value 46.347716
## iter  90 value 46.340328
## iter 100 value 46.337897
## final  value 46.337897 
## stopped after 100 iterations
## # weights:  19
## initial  value 234.829767 
## iter  10 value 73.014315
## iter  20 value 58.741330
## iter  30 value 56.247097
## iter  40 value 53.956790
## iter  50 value 53.656621
## iter  60 value 53.527428
## iter  70 value 53.507692
## iter  80 value 53.423070
## iter  90 value 53.421220
## iter 100 value 53.419734
## final  value 53.419734 
## stopped after 100 iterations
## # weights:  55
## initial  value 261.522122 
## iter  10 value 51.569936
## iter  20 value 40.113281
## iter  30 value 36.440547
## iter  40 value 33.357340
## iter  50 value 32.576208
## iter  60 value 32.315461
## iter  70 value 32.119885
## iter  80 value 32.059350
## iter  90 value 31.998957
## iter 100 value 31.968842
## final  value 31.968842 
## stopped after 100 iterations
## # weights:  91
## initial  value 281.261416 
## iter  10 value 60.817088
## iter  20 value 43.186530
## iter  30 value 32.541539
## iter  40 value 26.574356
## iter  50 value 24.160475
## iter  60 value 21.684184
## iter  70 value 20.395683
## iter  80 value 19.235389
## iter  90 value 18.969359
## iter 100 value 16.176906
## final  value 16.176906 
## stopped after 100 iterations
## # weights:  19
## initial  value 207.394153 
## iter  10 value 70.643302
## iter  20 value 59.681662
## iter  30 value 57.189521
## iter  40 value 52.219988
## iter  50 value 51.290159
## iter  60 value 51.266533
## iter  70 value 51.257408
## iter  80 value 51.255254
## iter  90 value 51.254050
## iter 100 value 51.253339
## final  value 51.253339 
## stopped after 100 iterations
## # weights:  55
## initial  value 236.801399 
## iter  10 value 61.379607
## iter  20 value 45.628953
## iter  30 value 42.068830
## iter  40 value 35.930905
## iter  50 value 34.052711
## iter  60 value 33.103166
## iter  70 value 32.869517
## iter  80 value 32.662327
## iter  90 value 32.361833
## iter 100 value 32.146702
## final  value 32.146702 
## stopped after 100 iterations
## # weights:  91
## initial  value 232.870679 
## iter  10 value 58.150468
## iter  20 value 36.402949
## iter  30 value 28.831657
## iter  40 value 26.345117
## iter  50 value 24.841441
## iter  60 value 24.266290
## iter  70 value 23.934294
## iter  80 value 23.881019
## iter  90 value 23.861180
## iter 100 value 23.853326
## final  value 23.853326 
## stopped after 100 iterations
## # weights:  19
## initial  value 196.435268 
## iter  10 value 71.822345
## iter  20 value 64.777617
## iter  30 value 64.135420
## final  value 64.117570 
## converged
## # weights:  55
## initial  value 242.977605 
## iter  10 value 68.106046
## iter  20 value 57.338613
## iter  30 value 53.363406
## iter  40 value 52.347199
## iter  50 value 51.814253
## iter  60 value 51.065032
## iter  70 value 50.124369
## iter  80 value 49.439191
## iter  90 value 49.372190
## iter 100 value 49.371466
## final  value 49.371466 
## stopped after 100 iterations
## # weights:  91
## initial  value 177.792951 
## iter  10 value 63.695869
## iter  20 value 54.946237
## iter  30 value 51.105713
## iter  40 value 50.052076
## iter  50 value 49.699651
## iter  60 value 49.573702
## iter  70 value 49.546419
## iter  80 value 49.541873
## iter  90 value 49.541570
## final  value 49.541552 
## converged
## # weights:  19
## initial  value 217.618502 
## iter  10 value 78.254102
## iter  20 value 60.114580
## iter  30 value 57.557933
## iter  40 value 55.772822
## iter  50 value 55.347920
## iter  60 value 55.169224
## iter  70 value 54.878204
## iter  80 value 54.783416
## iter  90 value 54.737038
## iter 100 value 54.587009
## final  value 54.587009 
## stopped after 100 iterations
## # weights:  55
## initial  value 183.784319 
## iter  10 value 64.731490
## iter  20 value 48.728212
## iter  30 value 42.660645
## iter  40 value 38.180583
## iter  50 value 35.928147
## iter  60 value 35.119640
## iter  70 value 34.848779
## iter  80 value 34.594598
## iter  90 value 34.457227
## iter 100 value 33.895486
## final  value 33.895486 
## stopped after 100 iterations
## # weights:  91
## initial  value 378.786346 
## iter  10 value 52.419706
## iter  20 value 29.883161
## iter  30 value 21.477103
## iter  40 value 19.100107
## iter  50 value 18.623419
## iter  60 value 18.334134
## iter  70 value 18.054271
## iter  80 value 17.835112
## iter  90 value 17.757448
## iter 100 value 17.671371
## final  value 17.671371 
## stopped after 100 iterations
## # weights:  19
## initial  value 202.467755 
## iter  10 value 74.465328
## iter  20 value 59.277939
## iter  30 value 58.045300
## iter  40 value 55.068215
## iter  50 value 54.049901
## iter  60 value 53.708362
## iter  70 value 53.483469
## iter  80 value 53.369995
## iter  90 value 53.344451
## iter 100 value 53.323855
## final  value 53.323855 
## stopped after 100 iterations
## # weights:  55
## initial  value 196.772788 
## iter  10 value 65.293154
## iter  20 value 53.482117
## iter  30 value 46.111869
## iter  40 value 42.881028
## iter  50 value 41.934715
## iter  60 value 41.791698
## iter  70 value 41.788300
## iter  80 value 41.784319
## iter  90 value 41.780836
## final  value 41.780786 
## converged
## # weights:  91
## initial  value 316.991702 
## iter  10 value 53.573340
## iter  20 value 33.698540
## iter  30 value 23.533359
## iter  40 value 20.999849
## iter  50 value 19.058390
## iter  60 value 18.848018
## iter  70 value 18.623635
## iter  80 value 18.444452
## iter  90 value 18.304483
## iter 100 value 18.224000
## final  value 18.224000 
## stopped after 100 iterations
## # weights:  19
## initial  value 218.728435 
## iter  10 value 82.731279
## iter  20 value 66.545404
## iter  30 value 63.431875
## iter  40 value 63.309413
## final  value 63.309410 
## converged
## # weights:  55
## initial  value 229.227220 
## iter  10 value 67.024856
## iter  20 value 56.342505
## iter  30 value 54.737296
## iter  40 value 54.585414
## iter  50 value 54.557621
## iter  60 value 54.555953
## final  value 54.555838 
## converged
## # weights:  91
## initial  value 295.317038 
## iter  10 value 66.544107
## iter  20 value 54.899180
## iter  30 value 50.618323
## iter  40 value 49.353798
## iter  50 value 48.579221
## iter  60 value 48.079477
## iter  70 value 47.764943
## iter  80 value 47.606177
## iter  90 value 47.585161
## iter 100 value 47.581422
## final  value 47.581422 
## stopped after 100 iterations
## # weights:  19
## initial  value 349.448924 
## iter  10 value 69.208816
## iter  20 value 58.899945
## iter  30 value 57.852741
## iter  40 value 55.121104
## iter  50 value 54.663346
## iter  60 value 54.512344
## iter  70 value 54.455280
## iter  80 value 54.413896
## iter  90 value 54.408472
## iter 100 value 54.402627
## final  value 54.402627 
## stopped after 100 iterations
## # weights:  55
## initial  value 195.207010 
## iter  10 value 50.071648
## iter  20 value 32.857156
## iter  30 value 27.318361
## iter  40 value 26.868216
## iter  50 value 26.576187
## iter  60 value 26.407138
## iter  70 value 26.351063
## iter  80 value 26.311904
## iter  90 value 26.284679
## iter 100 value 26.262165
## final  value 26.262165 
## stopped after 100 iterations
## # weights:  91
## initial  value 308.015371 
## iter  10 value 54.478756
## iter  20 value 32.704219
## iter  30 value 24.789417
## iter  40 value 20.092643
## iter  50 value 17.992633
## iter  60 value 16.505467
## iter  70 value 14.847475
## iter  80 value 13.897023
## iter  90 value 13.572274
## iter 100 value 13.471947
## final  value 13.471947 
## stopped after 100 iterations
## # weights:  19
## initial  value 246.993544 
## iter  10 value 64.854400
## iter  20 value 50.831617
## iter  30 value 49.190357
## iter  40 value 48.653893
## iter  50 value 45.938731
## iter  60 value 43.462624
## final  value 43.452623 
## converged
## # weights:  55
## initial  value 363.282004 
## iter  10 value 76.912813
## iter  20 value 50.163897
## iter  30 value 36.187624
## iter  40 value 31.840189
## iter  50 value 29.355228
## iter  60 value 27.350450
## iter  70 value 26.921066
## iter  80 value 26.795160
## iter  90 value 26.760821
## iter 100 value 26.725126
## final  value 26.725126 
## stopped after 100 iterations
## # weights:  91
## initial  value 242.132643 
## iter  10 value 44.018221
## iter  20 value 26.980971
## iter  30 value 17.122800
## iter  40 value 13.879825
## iter  50 value 12.878408
## iter  60 value 12.227754
## iter  70 value 11.907123
## iter  80 value 11.830168
## iter  90 value 11.826454
## iter  90 value 11.826454
## iter  90 value 11.826454
## final  value 11.826454 
## converged
## # weights:  19
## initial  value 386.993634 
## iter  10 value 74.225034
## iter  20 value 60.485415
## iter  30 value 58.237655
## iter  40 value 58.144830
## final  value 58.144794 
## converged
## # weights:  55
## initial  value 357.063125 
## iter  10 value 61.643764
## iter  20 value 50.768323
## iter  30 value 47.630270
## iter  40 value 47.256482
## iter  50 value 47.190805
## final  value 47.186377 
## converged
## # weights:  91
## initial  value 173.103810 
## iter  10 value 58.160232
## iter  20 value 48.402269
## iter  30 value 44.975413
## iter  40 value 44.259588
## iter  50 value 43.854557
## iter  60 value 43.406943
## iter  70 value 43.249164
## iter  80 value 43.111895
## iter  90 value 43.096889
## iter 100 value 43.096250
## final  value 43.096250 
## stopped after 100 iterations
## # weights:  19
## initial  value 267.733736 
## iter  10 value 57.809824
## iter  20 value 48.559612
## iter  30 value 47.099040
## iter  40 value 43.725276
## iter  50 value 43.604019
## iter  60 value 43.474013
## iter  70 value 43.450353
## iter  80 value 43.435979
## iter  90 value 43.432528
## iter 100 value 43.430993
## final  value 43.430993 
## stopped after 100 iterations
## # weights:  55
## initial  value 162.802360 
## iter  10 value 56.896811
## iter  20 value 41.122090
## iter  30 value 34.489656
## iter  40 value 31.019596
## iter  50 value 29.035210
## iter  60 value 27.425231
## iter  70 value 25.438924
## iter  80 value 21.241751
## iter  90 value 19.006461
## iter 100 value 18.084083
## final  value 18.084083 
## stopped after 100 iterations
## # weights:  91
## initial  value 268.422111 
## iter  10 value 41.424717
## iter  20 value 25.821627
## iter  30 value 18.530745
## iter  40 value 16.511387
## iter  50 value 15.730457
## iter  60 value 15.535561
## iter  70 value 15.285947
## iter  80 value 14.975382
## iter  90 value 14.525232
## iter 100 value 14.352778
## final  value 14.352778 
## stopped after 100 iterations
## # weights:  19
## initial  value 335.485174 
## iter  10 value 88.786917
## iter  20 value 72.994772
## iter  30 value 70.186378
## iter  40 value 69.179882
## iter  50 value 68.740282
## iter  60 value 68.309508
## iter  70 value 68.090210
## iter  80 value 67.957336
## iter  90 value 67.906911
## iter 100 value 67.835956
## final  value 67.835956 
## stopped after 100 iterations
## # weights:  55
## initial  value 246.591173 
## iter  10 value 69.450271
## iter  20 value 39.853947
## iter  30 value 32.466321
## iter  40 value 30.571878
## iter  50 value 29.883769
## iter  60 value 28.908355
## iter  70 value 28.411610
## iter  80 value 28.358369
## iter  90 value 28.332736
## iter 100 value 28.323511
## final  value 28.323511 
## stopped after 100 iterations
## # weights:  91
## initial  value 239.013890 
## iter  10 value 58.820510
## iter  20 value 45.325140
## iter  30 value 30.496884
## iter  40 value 19.914527
## iter  50 value 17.891952
## iter  60 value 17.650758
## iter  70 value 17.636494
## iter  80 value 17.618980
## iter  90 value 17.614161
## iter 100 value 17.612108
## final  value 17.612108 
## stopped after 100 iterations
## # weights:  19
## initial  value 395.079760 
## iter  10 value 89.270756
## iter  20 value 71.086782
## iter  30 value 66.752067
## iter  40 value 66.580751
## iter  50 value 66.578693
## final  value 66.578573 
## converged
## # weights:  55
## initial  value 425.717668 
## iter  10 value 72.045461
## iter  20 value 63.907918
## iter  30 value 61.481175
## iter  40 value 59.120582
## iter  50 value 57.522377
## iter  60 value 57.094400
## iter  70 value 55.193180
## iter  80 value 52.794128
## iter  90 value 52.157176
## iter 100 value 52.129110
## final  value 52.129110 
## stopped after 100 iterations
## # weights:  91
## initial  value 437.131068 
## iter  10 value 69.438174
## iter  20 value 54.590007
## iter  30 value 50.646110
## iter  40 value 49.858702
## iter  50 value 49.293383
## iter  60 value 48.827176
## iter  70 value 48.599183
## iter  80 value 48.460784
## iter  90 value 48.242940
## iter 100 value 48.179829
## final  value 48.179829 
## stopped after 100 iterations
## # weights:  19
## initial  value 298.194284 
## iter  10 value 74.782861
## iter  20 value 62.257616
## iter  30 value 59.877646
## iter  40 value 56.697471
## iter  50 value 55.096987
## iter  60 value 53.140371
## iter  70 value 50.923179
## iter  80 value 50.818069
## iter  90 value 50.654575
## iter 100 value 50.641381
## final  value 50.641381 
## stopped after 100 iterations
## # weights:  55
## initial  value 274.434827 
## iter  10 value 60.915074
## iter  20 value 37.323125
## iter  30 value 34.602924
## iter  40 value 34.011539
## iter  50 value 33.821015
## iter  60 value 33.568655
## iter  70 value 33.411168
## iter  80 value 33.373966
## iter  90 value 33.363890
## iter 100 value 33.356627
## final  value 33.356627 
## stopped after 100 iterations
## # weights:  91
## initial  value 212.232202 
## iter  10 value 54.080758
## iter  20 value 32.769799
## iter  30 value 28.555251
## iter  40 value 20.270780
## iter  50 value 17.741457
## iter  60 value 15.314122
## iter  70 value 14.617242
## iter  80 value 14.194513
## iter  90 value 13.844330
## iter 100 value 13.763050
## final  value 13.763050 
## stopped after 100 iterations
## # weights:  19
## initial  value 292.726609 
## iter  10 value 69.913629
## iter  20 value 60.671262
## iter  30 value 59.551713
## iter  40 value 54.510130
## iter  50 value 53.099590
## iter  60 value 53.049305
## iter  70 value 53.031408
## iter  80 value 53.025609
## iter  90 value 53.021245
## iter 100 value 53.015743
## final  value 53.015743 
## stopped after 100 iterations
## # weights:  55
## initial  value 375.186805 
## iter  10 value 53.939380
## iter  20 value 44.308319
## iter  30 value 36.492661
## iter  40 value 34.396573
## iter  50 value 32.822644
## iter  60 value 31.792674
## iter  70 value 31.242835
## iter  80 value 30.871310
## iter  90 value 30.583173
## iter 100 value 30.318773
## final  value 30.318773 
## stopped after 100 iterations
## # weights:  91
## initial  value 368.005445 
## iter  10 value 51.908749
## iter  20 value 28.844593
## iter  30 value 22.949580
## iter  40 value 21.293289
## iter  50 value 20.460648
## iter  60 value 19.922601
## iter  70 value 19.853200
## iter  80 value 19.821346
## final  value 19.821152 
## converged
## # weights:  19
## initial  value 202.900189 
## iter  10 value 77.840766
## iter  20 value 66.982855
## iter  30 value 63.774393
## iter  40 value 63.722049
## iter  40 value 63.722049
## final  value 63.722049 
## converged
## # weights:  55
## initial  value 232.610337 
## iter  10 value 67.132294
## iter  20 value 56.496969
## iter  30 value 53.451045
## iter  40 value 52.990462
## iter  50 value 52.513235
## iter  60 value 52.479696
## final  value 52.479281 
## converged
## # weights:  91
## initial  value 201.759972 
## iter  10 value 64.826158
## iter  20 value 56.088533
## iter  30 value 53.149304
## iter  40 value 50.475047
## iter  50 value 48.174399
## iter  60 value 47.500128
## iter  70 value 47.350342
## iter  80 value 47.345291
## iter  90 value 47.345245
## final  value 47.345240 
## converged
## # weights:  19
## initial  value 231.272813 
## iter  10 value 79.426334
## iter  20 value 67.902903
## iter  30 value 60.321314
## iter  40 value 56.750669
## iter  50 value 54.757981
## iter  60 value 54.151059
## iter  70 value 54.097815
## iter  80 value 54.007340
## iter  90 value 53.943353
## iter 100 value 53.921854
## final  value 53.921854 
## stopped after 100 iterations
## # weights:  55
## initial  value 237.133547 
## iter  10 value 75.079962
## iter  20 value 53.155182
## iter  30 value 47.933497
## iter  40 value 45.056141
## iter  50 value 43.108208
## iter  60 value 42.258520
## iter  70 value 41.493179
## iter  80 value 41.055758
## iter  90 value 38.888900
## iter 100 value 36.960453
## final  value 36.960453 
## stopped after 100 iterations
## # weights:  91
## initial  value 304.133481 
## iter  10 value 56.187131
## iter  20 value 41.816169
## iter  30 value 32.720014
## iter  40 value 31.615542
## iter  50 value 30.843557
## iter  60 value 29.440065
## iter  70 value 27.536155
## iter  80 value 27.131743
## iter  90 value 26.765345
## iter 100 value 25.620792
## final  value 25.620792 
## stopped after 100 iterations
## # weights:  19
## initial  value 176.659804 
## iter  10 value 74.746276
## iter  20 value 58.712423
## iter  30 value 55.977648
## iter  40 value 55.691848
## iter  50 value 55.691301
## iter  60 value 55.690993
## final  value 55.690988 
## converged
## # weights:  55
## initial  value 205.389557 
## iter  10 value 52.897015
## iter  20 value 42.388630
## iter  30 value 38.892629
## iter  40 value 38.262428
## iter  50 value 38.057104
## iter  60 value 37.860433
## iter  70 value 37.501500
## iter  80 value 37.359268
## iter  90 value 37.206188
## iter 100 value 37.095502
## final  value 37.095502 
## stopped after 100 iterations
## # weights:  91
## initial  value 177.626786 
## iter  10 value 52.740711
## iter  20 value 34.602275
## iter  30 value 24.809587
## iter  40 value 18.516754
## iter  50 value 17.827873
## iter  60 value 17.721335
## iter  70 value 17.618842
## iter  80 value 17.596688
## iter  90 value 17.583429
## iter 100 value 17.415425
## final  value 17.415425 
## stopped after 100 iterations
## # weights:  19
## initial  value 252.638539 
## iter  10 value 77.422512
## iter  20 value 65.516822
## iter  30 value 64.253318
## final  value 64.248237 
## converged
## # weights:  55
## initial  value 333.153552 
## iter  10 value 77.709091
## iter  20 value 55.422291
## iter  30 value 51.162756
## iter  40 value 50.539515
## iter  50 value 50.120923
## iter  60 value 49.868815
## iter  70 value 49.828661
## final  value 49.828660 
## converged
## # weights:  91
## initial  value 254.399185 
## iter  10 value 68.990007
## iter  20 value 53.830781
## iter  30 value 50.517721
## iter  40 value 49.629947
## iter  50 value 48.626228
## iter  60 value 48.434094
## iter  70 value 48.341156
## iter  80 value 48.319947
## iter  90 value 48.312915
## final  value 48.312872 
## converged
## # weights:  19
## initial  value 268.054208 
## iter  10 value 81.070256
## iter  20 value 62.621370
## iter  30 value 59.153959
## iter  40 value 58.803400
## iter  50 value 56.657510
## iter  60 value 53.646139
## iter  70 value 51.090887
## iter  80 value 47.887358
## iter  90 value 47.814254
## iter 100 value 47.707347
## final  value 47.707347 
## stopped after 100 iterations
## # weights:  55
## initial  value 290.680831 
## iter  10 value 57.417080
## iter  20 value 44.541535
## iter  30 value 39.008722
## iter  40 value 34.547353
## iter  50 value 34.047702
## iter  60 value 33.817719
## iter  70 value 33.744820
## iter  80 value 33.651535
## iter  90 value 33.577566
## iter 100 value 33.491793
## final  value 33.491793 
## stopped after 100 iterations
## # weights:  91
## initial  value 301.228360 
## iter  10 value 59.377766
## iter  20 value 41.227684
## iter  30 value 35.890780
## iter  40 value 29.665137
## iter  50 value 21.087438
## iter  60 value 18.401496
## iter  70 value 18.056371
## iter  80 value 17.744188
## iter  90 value 17.522298
## iter 100 value 17.272291
## final  value 17.272291 
## stopped after 100 iterations
## # weights:  19
## initial  value 308.510267 
## iter  10 value 65.175531
## iter  20 value 57.156403
## iter  30 value 53.838251
## iter  40 value 50.757632
## iter  50 value 45.887542
## iter  60 value 42.102801
## iter  70 value 42.089890
## final  value 42.089703 
## converged
## # weights:  55
## initial  value 506.865483 
## iter  10 value 56.454472
## iter  20 value 43.933651
## iter  30 value 34.364813
## iter  40 value 25.631085
## iter  50 value 22.448985
## iter  60 value 22.260628
## iter  70 value 22.157452
## iter  80 value 22.151375
## iter  90 value 22.132775
## iter 100 value 22.069637
## final  value 22.069637 
## stopped after 100 iterations
## # weights:  91
## initial  value 246.560998 
## iter  10 value 47.404136
## iter  20 value 33.095846
## iter  30 value 26.768508
## iter  40 value 24.783396
## iter  50 value 24.189613
## iter  60 value 23.934137
## iter  70 value 23.608166
## iter  80 value 23.391239
## iter  90 value 22.605924
## iter 100 value 20.563030
## final  value 20.563030 
## stopped after 100 iterations
## # weights:  19
## initial  value 233.928529 
## iter  10 value 81.987972
## iter  20 value 65.106444
## iter  30 value 63.242023
## iter  40 value 63.086571
## final  value 63.086550 
## converged
## # weights:  55
## initial  value 419.993554 
## iter  10 value 74.639699
## iter  20 value 61.248177
## iter  30 value 55.540710
## iter  40 value 54.054726
## iter  50 value 53.731017
## iter  60 value 53.607004
## iter  70 value 53.513178
## iter  80 value 53.501086
## final  value 53.501057 
## converged
## # weights:  91
## initial  value 575.569912 
## iter  10 value 68.409659
## iter  20 value 54.144896
## iter  30 value 52.063758
## iter  40 value 51.197946
## iter  50 value 50.772029
## iter  60 value 50.509610
## iter  70 value 49.968309
## iter  80 value 49.340129
## iter  90 value 49.171858
## iter 100 value 49.164422
## final  value 49.164422 
## stopped after 100 iterations
## # weights:  19
## initial  value 326.418925 
## iter  10 value 71.496288
## iter  20 value 58.906654
## iter  30 value 56.879382
## iter  40 value 55.397204
## iter  50 value 54.653339
## iter  60 value 53.961206
## iter  70 value 53.827769
## iter  80 value 53.808568
## iter  90 value 53.802250
## iter 100 value 53.796787
## final  value 53.796787 
## stopped after 100 iterations
## # weights:  55
## initial  value 356.966037 
## iter  10 value 57.481627
## iter  20 value 47.387121
## iter  30 value 42.258523
## iter  40 value 41.182159
## iter  50 value 41.086212
## iter  60 value 41.046748
## iter  70 value 41.014348
## iter  80 value 40.890582
## iter  90 value 40.752865
## iter 100 value 40.690041
## final  value 40.690041 
## stopped after 100 iterations
## # weights:  91
## initial  value 433.221101 
## iter  10 value 58.853868
## iter  20 value 37.244193
## iter  30 value 31.042684
## iter  40 value 29.657283
## iter  50 value 29.245143
## iter  60 value 29.162437
## iter  70 value 29.117516
## iter  80 value 29.055515
## iter  90 value 29.030668
## iter 100 value 28.991216
## final  value 28.991216 
## stopped after 100 iterations
## # weights:  19
## initial  value 226.359644 
## iter  10 value 83.421895
## iter  20 value 57.601544
## iter  30 value 55.636622
## iter  40 value 51.539011
## iter  50 value 50.996333
## iter  60 value 50.986604
## iter  70 value 50.976768
## iter  80 value 50.975552
## iter  90 value 50.975008
## iter 100 value 50.973890
## final  value 50.973890 
## stopped after 100 iterations
## # weights:  55
## initial  value 254.606484 
## iter  10 value 55.796377
## iter  20 value 43.282493
## iter  30 value 30.808571
## iter  40 value 25.870439
## iter  50 value 24.340801
## iter  60 value 21.542650
## iter  70 value 21.307249
## iter  80 value 21.076943
## iter  90 value 20.978079
## iter 100 value 20.891097
## final  value 20.891097 
## stopped after 100 iterations
## # weights:  91
## initial  value 266.687156 
## iter  10 value 52.766730
## iter  20 value 34.290241
## iter  30 value 18.394648
## iter  40 value 15.739678
## iter  50 value 14.757027
## iter  60 value 14.473370
## iter  70 value 14.430354
## iter  80 value 14.426960
## iter  90 value 14.425404
## iter 100 value 14.424261
## final  value 14.424261 
## stopped after 100 iterations
## # weights:  19
## initial  value 243.784791 
## iter  10 value 69.420989
## iter  20 value 63.632686
## iter  30 value 63.176734
## final  value 63.175660 
## converged
## # weights:  55
## initial  value 231.790367 
## iter  10 value 66.962946
## iter  20 value 54.097152
## iter  30 value 48.524278
## iter  40 value 48.054175
## iter  50 value 47.938968
## iter  60 value 47.936764
## final  value 47.936697 
## converged
## # weights:  91
## initial  value 225.554503 
## iter  10 value 64.249536
## iter  20 value 49.836253
## iter  30 value 45.149617
## iter  40 value 43.173307
## iter  50 value 42.860880
## iter  60 value 42.765625
## iter  70 value 42.748458
## iter  80 value 42.745683
## iter  90 value 42.745641
## final  value 42.745640 
## converged
## # weights:  19
## initial  value 376.020243 
## iter  10 value 72.785470
## iter  20 value 56.933482
## iter  30 value 55.552897
## iter  40 value 54.286446
## iter  50 value 54.103831
## iter  60 value 53.996203
## iter  70 value 53.336708
## iter  80 value 53.018497
## iter  90 value 53.014349
## final  value 53.003529 
## converged
## # weights:  55
## initial  value 228.597390 
## iter  10 value 52.531193
## iter  20 value 39.977729
## iter  30 value 33.029947
## iter  40 value 32.095126
## iter  50 value 31.611059
## iter  60 value 31.537681
## iter  70 value 31.494439
## iter  80 value 31.314692
## iter  90 value 31.014655
## iter 100 value 30.977082
## final  value 30.977082 
## stopped after 100 iterations
## # weights:  91
## initial  value 389.519148 
## iter  10 value 46.969750
## iter  20 value 25.260254
## iter  30 value 16.263974
## iter  40 value 15.032825
## iter  50 value 14.794538
## iter  60 value 14.607409
## iter  70 value 14.506943
## iter  80 value 14.426376
## iter  90 value 14.332939
## iter 100 value 12.320685
## final  value 12.320685 
## stopped after 100 iterations
## # weights:  19
## initial  value 404.435684 
## iter  10 value 63.851592
## iter  20 value 47.622620
## iter  30 value 45.708615
## iter  40 value 41.289196
## iter  50 value 41.161106
## iter  60 value 41.079461
## iter  70 value 41.037109
## iter  80 value 41.003518
## iter  90 value 40.991750
## iter 100 value 40.985880
## final  value 40.985880 
## stopped after 100 iterations
## # weights:  55
## initial  value 445.622516 
## iter  10 value 62.288557
## iter  20 value 37.581901
## iter  30 value 32.008241
## iter  40 value 30.865928
## iter  50 value 29.833520
## iter  60 value 29.305165
## iter  70 value 29.031236
## iter  80 value 29.023545
## iter  90 value 29.013768
## iter 100 value 28.864291
## final  value 28.864291 
## stopped after 100 iterations
## # weights:  91
## initial  value 299.653375 
## iter  10 value 56.076666
## iter  20 value 36.684342
## iter  30 value 16.573535
## iter  40 value 11.523634
## iter  50 value 10.960363
## iter  60 value 10.835304
## iter  70 value 10.799262
## iter  80 value 10.777075
## iter  90 value 10.751904
## iter 100 value 10.739397
## final  value 10.739397 
## stopped after 100 iterations
## # weights:  19
## initial  value 256.771574 
## iter  10 value 81.351277
## iter  20 value 63.035346
## iter  30 value 59.753031
## iter  40 value 59.568053
## final  value 59.568049 
## converged
## # weights:  55
## initial  value 246.745356 
## iter  10 value 67.535588
## iter  20 value 53.520296
## iter  30 value 50.410631
## iter  40 value 49.903607
## iter  50 value 49.834180
## iter  60 value 49.715581
## iter  70 value 49.650781
## iter  80 value 49.619211
## iter  90 value 49.615374
## final  value 49.615234 
## converged
## # weights:  91
## initial  value 265.788969 
## iter  10 value 62.393216
## iter  20 value 48.888391
## iter  30 value 46.389413
## iter  40 value 45.842192
## iter  50 value 45.466308
## iter  60 value 45.372780
## iter  70 value 45.275407
## iter  80 value 45.219046
## iter  90 value 44.883669
## iter 100 value 44.813713
## final  value 44.813713 
## stopped after 100 iterations
## # weights:  19
## initial  value 256.849167 
## iter  10 value 70.630037
## iter  20 value 62.484450
## iter  30 value 51.678077
## iter  40 value 49.903268
## iter  50 value 47.723206
## iter  60 value 47.668103
## iter  70 value 47.477461
## iter  80 value 47.421828
## iter  90 value 47.405091
## iter 100 value 47.404375
## final  value 47.404375 
## stopped after 100 iterations
## # weights:  55
## initial  value 175.720942 
## iter  10 value 62.543955
## iter  20 value 49.125390
## iter  30 value 36.156279
## iter  40 value 28.030350
## iter  50 value 27.121846
## iter  60 value 26.950689
## iter  70 value 26.822914
## iter  80 value 26.781882
## iter  90 value 26.766154
## iter 100 value 26.743199
## final  value 26.743199 
## stopped after 100 iterations
## # weights:  91
## initial  value 223.327952 
## iter  10 value 46.047272
## iter  20 value 25.873462
## iter  30 value 18.601254
## iter  40 value 13.071544
## iter  50 value 12.298073
## iter  60 value 11.922616
## iter  70 value 11.678080
## iter  80 value 11.368248
## iter  90 value 11.133273
## iter 100 value 10.970552
## final  value 10.970552 
## stopped after 100 iterations
## # weights:  19
## initial  value 330.969961 
## iter  10 value 87.058372
## iter  20 value 73.237998
## iter  30 value 65.447819
## iter  40 value 61.160233
## iter  50 value 57.332805
## iter  60 value 55.797939
## iter  70 value 54.725563
## iter  80 value 54.277106
## iter  90 value 54.127989
## iter 100 value 53.207594
## final  value 53.207594 
## stopped after 100 iterations
## # weights:  55
## initial  value 351.819312 
## iter  10 value 67.400613
## iter  20 value 50.579067
## iter  30 value 46.302666
## iter  40 value 43.765555
## iter  50 value 41.774631
## iter  60 value 40.852661
## iter  70 value 39.840004
## iter  80 value 38.889692
## iter  90 value 36.209944
## iter 100 value 35.226811
## final  value 35.226811 
## stopped after 100 iterations
## # weights:  91
## initial  value 203.973083 
## iter  10 value 51.255658
## iter  20 value 34.004862
## iter  30 value 22.665597
## iter  40 value 20.020491
## iter  50 value 17.655464
## iter  60 value 17.256489
## iter  70 value 17.118147
## iter  80 value 16.982694
## iter  90 value 16.945811
## iter 100 value 16.929445
## final  value 16.929445 
## stopped after 100 iterations
## # weights:  19
## initial  value 181.877420 
## iter  10 value 88.790427
## iter  20 value 71.693908
## iter  30 value 68.015926
## iter  40 value 67.374472
## iter  50 value 67.368610
## final  value 67.368403 
## converged
## # weights:  55
## initial  value 401.717217 
## iter  10 value 74.795276
## iter  20 value 61.114334
## iter  30 value 56.293936
## iter  40 value 54.769175
## iter  50 value 54.353985
## iter  60 value 54.348155
## final  value 54.348076 
## converged
## # weights:  91
## initial  value 308.006191 
## iter  10 value 65.322943
## iter  20 value 55.614159
## iter  30 value 52.138236
## iter  40 value 51.458945
## iter  50 value 50.703791
## iter  60 value 50.329745
## iter  70 value 49.531731
## iter  80 value 49.094528
## iter  90 value 48.967336
## iter 100 value 48.858398
## final  value 48.858398 
## stopped after 100 iterations
## # weights:  19
## initial  value 180.350025 
## iter  10 value 83.818034
## iter  20 value 65.183664
## iter  30 value 60.543547
## iter  40 value 59.438830
## iter  50 value 59.271991
## iter  60 value 58.542066
## iter  70 value 58.295300
## iter  80 value 58.242745
## iter  90 value 58.222294
## iter 100 value 58.214114
## final  value 58.214114 
## stopped after 100 iterations
## # weights:  55
## initial  value 236.032368 
## iter  10 value 55.506492
## iter  20 value 46.597901
## iter  30 value 40.327944
## iter  40 value 39.742530
## iter  50 value 39.528536
## iter  60 value 39.452433
## iter  70 value 39.357597
## iter  80 value 39.152169
## iter  90 value 39.083448
## iter 100 value 39.032604
## final  value 39.032604 
## stopped after 100 iterations
## # weights:  91
## initial  value 330.569032 
## iter  10 value 74.512851
## iter  20 value 50.084702
## iter  30 value 36.491565
## iter  40 value 28.558547
## iter  50 value 25.719708
## iter  60 value 23.523773
## iter  70 value 21.287419
## iter  80 value 20.560929
## iter  90 value 19.819595
## iter 100 value 18.458991
## final  value 18.458991 
## stopped after 100 iterations
## # weights:  19
## initial  value 300.324998 
## iter  10 value 80.204794
## iter  20 value 52.285119
## iter  30 value 50.654482
## iter  40 value 49.939125
## iter  50 value 48.993752
## iter  60 value 43.350994
## iter  70 value 42.091219
## final  value 42.088122 
## converged
## # weights:  55
## initial  value 211.541823 
## iter  10 value 50.573964
## iter  20 value 30.954622
## iter  30 value 24.808440
## iter  40 value 22.942432
## iter  50 value 22.429798
## iter  60 value 22.428226
## iter  70 value 22.427580
## final  value 22.427578 
## converged
## # weights:  91
## initial  value 424.472139 
## iter  10 value 48.535058
## iter  20 value 29.211769
## iter  30 value 22.570950
## iter  40 value 17.612961
## iter  50 value 17.075583
## iter  60 value 16.324980
## iter  70 value 14.809099
## iter  80 value 14.775694
## iter  90 value 14.763743
## iter 100 value 14.743411
## final  value 14.743411 
## stopped after 100 iterations
## # weights:  19
## initial  value 273.126146 
## iter  10 value 78.188293
## iter  20 value 63.997637
## iter  30 value 60.622828
## iter  40 value 60.546126
## iter  50 value 60.545315
## final  value 60.545287 
## converged
## # weights:  55
## initial  value 257.580057 
## iter  10 value 64.791838
## iter  20 value 52.603570
## iter  30 value 49.888547
## iter  40 value 49.631610
## iter  50 value 49.539166
## iter  60 value 49.531256
## final  value 49.531115 
## converged
## # weights:  91
## initial  value 207.755418 
## iter  10 value 62.133957
## iter  20 value 51.204955
## iter  30 value 47.585307
## iter  40 value 46.599759
## iter  50 value 45.944210
## iter  60 value 45.604309
## iter  70 value 45.534341
## iter  80 value 45.404186
## iter  90 value 45.136840
## iter 100 value 45.082250
## final  value 45.082250 
## stopped after 100 iterations
## # weights:  19
## initial  value 295.598643 
## iter  10 value 76.064918
## iter  20 value 67.872210
## iter  30 value 59.553205
## iter  40 value 54.567292
## iter  50 value 53.962741
## iter  60 value 53.370415
## iter  70 value 52.505679
## iter  80 value 51.380738
## iter  90 value 50.819594
## iter 100 value 50.620348
## final  value 50.620348 
## stopped after 100 iterations
## # weights:  55
## initial  value 248.420492 
## iter  10 value 52.585959
## iter  20 value 35.484848
## iter  30 value 29.493675
## iter  40 value 26.707059
## iter  50 value 26.168102
## iter  60 value 25.367729
## iter  70 value 24.965798
## iter  80 value 24.927108
## iter  90 value 24.900908
## iter 100 value 24.882545
## final  value 24.882545 
## stopped after 100 iterations
## # weights:  91
## initial  value 218.943635 
## iter  10 value 49.409875
## iter  20 value 29.449084
## iter  30 value 18.224624
## iter  40 value 16.297601
## iter  50 value 15.571202
## iter  60 value 15.388191
## iter  70 value 15.207332
## iter  80 value 15.001048
## iter  90 value 14.864986
## iter 100 value 14.636364
## final  value 14.636364 
## stopped after 100 iterations
## # weights:  19
## initial  value 207.996285 
## iter  10 value 79.077360
## iter  20 value 64.844734
## iter  30 value 61.736270
## iter  40 value 58.752446
## iter  50 value 54.508072
## iter  60 value 50.378106
## iter  70 value 48.015654
## iter  80 value 47.671427
## iter  90 value 46.425201
## iter 100 value 44.812281
## final  value 44.812281 
## stopped after 100 iterations
## # weights:  55
## initial  value 222.134792 
## iter  10 value 68.089802
## iter  20 value 43.416495
## iter  30 value 38.311202
## iter  40 value 35.733371
## iter  50 value 32.636071
## iter  60 value 32.250347
## iter  70 value 31.976219
## iter  80 value 31.755813
## iter  90 value 31.224638
## iter 100 value 30.782517
## final  value 30.782517 
## stopped after 100 iterations
## # weights:  91
## initial  value 382.742903 
## iter  10 value 52.238874
## iter  20 value 33.057543
## iter  30 value 28.686582
## iter  40 value 24.870190
## iter  50 value 20.379668
## iter  60 value 19.904424
## iter  70 value 18.840900
## iter  80 value 18.219500
## iter  90 value 17.775244
## iter 100 value 17.754622
## final  value 17.754622 
## stopped after 100 iterations
## # weights:  19
## initial  value 252.967623 
## iter  10 value 82.616214
## iter  20 value 69.082069
## iter  30 value 65.044497
## iter  40 value 62.764979
## iter  50 value 62.764684
## final  value 62.764664 
## converged
## # weights:  55
## initial  value 409.580343 
## iter  10 value 71.207776
## iter  20 value 60.416250
## iter  30 value 56.803928
## iter  40 value 53.448736
## iter  50 value 52.826447
## iter  60 value 51.262775
## iter  70 value 51.078774
## iter  80 value 51.045530
## iter  90 value 51.023533
## iter 100 value 50.893222
## final  value 50.893222 
## stopped after 100 iterations
## # weights:  91
## initial  value 223.144644 
## iter  10 value 68.669361
## iter  20 value 56.880986
## iter  30 value 49.101580
## iter  40 value 47.648117
## iter  50 value 47.056871
## iter  60 value 46.861067
## iter  70 value 46.831034
## iter  80 value 46.808539
## iter  90 value 46.788499
## iter 100 value 46.786652
## final  value 46.786652 
## stopped after 100 iterations
## # weights:  19
## initial  value 421.601703 
## iter  10 value 95.521194
## iter  20 value 65.003727
## iter  30 value 59.000571
## iter  40 value 57.956878
## iter  50 value 56.204125
## iter  60 value 52.668580
## iter  70 value 49.722915
## iter  80 value 47.629099
## iter  90 value 47.382124
## iter 100 value 47.194624
## final  value 47.194624 
## stopped after 100 iterations
## # weights:  55
## initial  value 303.759753 
## iter  10 value 66.133985
## iter  20 value 51.713646
## iter  30 value 42.057181
## iter  40 value 38.494273
## iter  50 value 36.144959
## iter  60 value 35.394672
## iter  70 value 35.204228
## iter  80 value 35.090386
## iter  90 value 35.049038
## iter 100 value 35.019636
## final  value 35.019636 
## stopped after 100 iterations
## # weights:  91
## initial  value 225.755716 
## iter  10 value 55.774191
## iter  20 value 32.609176
## iter  30 value 28.111265
## iter  40 value 26.032517
## iter  50 value 25.423900
## iter  60 value 24.814228
## iter  70 value 24.243204
## iter  80 value 24.056139
## iter  90 value 23.488186
## iter 100 value 22.366168
## final  value 22.366168 
## stopped after 100 iterations
## # weights:  19
## initial  value 401.298318 
## iter  10 value 111.264475
## iter  20 value 91.801809
## iter  30 value 90.645145
## iter  40 value 86.835528
## iter  50 value 72.395869
## iter  60 value 62.097038
## iter  70 value 52.377797
## iter  80 value 51.444460
## iter  90 value 51.194684
## iter 100 value 47.835794
## final  value 47.835794 
## stopped after 100 iterations
## # weights:  55
## initial  value 162.605581 
## iter  10 value 46.271786
## iter  20 value 34.005835
## iter  30 value 31.191578
## iter  40 value 29.383663
## iter  50 value 29.084689
## iter  60 value 28.975279
## iter  70 value 28.974235
## final  value 28.921651 
## converged
## # weights:  91
## initial  value 426.407712 
## iter  10 value 49.153279
## iter  20 value 25.129812
## iter  30 value 17.647181
## iter  40 value 15.066686
## iter  50 value 13.814019
## iter  60 value 13.539550
## iter  70 value 13.470047
## iter  80 value 13.323907
## iter  90 value 13.287563
## iter 100 value 13.284517
## final  value 13.284517 
## stopped after 100 iterations
## # weights:  19
## initial  value 247.630632 
## iter  10 value 77.398400
## iter  20 value 63.492406
## iter  30 value 62.303338
## iter  40 value 62.270679
## iter  40 value 62.270679
## final  value 62.270679 
## converged
## # weights:  55
## initial  value 347.433601 
## iter  10 value 66.926217
## iter  20 value 55.816776
## iter  30 value 53.511411
## iter  40 value 52.316689
## iter  50 value 51.850208
## iter  60 value 51.638296
## iter  70 value 51.635878
## final  value 51.635746 
## converged
## # weights:  91
## initial  value 215.048558 
## iter  10 value 63.710233
## iter  20 value 53.732687
## iter  30 value 51.596878
## iter  40 value 49.641423
## iter  50 value 48.472978
## iter  60 value 46.661905
## iter  70 value 46.099980
## iter  80 value 45.699502
## iter  90 value 45.578829
## iter 100 value 45.577084
## final  value 45.577084 
## stopped after 100 iterations
## # weights:  19
## initial  value 415.818633 
## iter  10 value 119.695298
## iter  20 value 97.058014
## iter  30 value 91.631151
## iter  40 value 90.113678
## iter  50 value 83.703949
## iter  60 value 74.254033
## iter  70 value 62.653395
## iter  80 value 53.627920
## iter  90 value 52.193756
## iter 100 value 51.069083
## final  value 51.069083 
## stopped after 100 iterations
## # weights:  55
## initial  value 231.403628 
## iter  10 value 73.251443
## iter  20 value 43.959602
## iter  30 value 34.211322
## iter  40 value 29.012719
## iter  50 value 26.707419
## iter  60 value 22.147226
## iter  70 value 21.202795
## iter  80 value 20.040647
## iter  90 value 19.300483
## iter 100 value 18.952596
## final  value 18.952596 
## stopped after 100 iterations
## # weights:  91
## initial  value 170.565488 
## iter  10 value 50.790114
## iter  20 value 32.266398
## iter  30 value 22.199762
## iter  40 value 19.869619
## iter  50 value 19.065767
## iter  60 value 18.367916
## iter  70 value 18.054521
## iter  80 value 17.833111
## iter  90 value 17.714587
## iter 100 value 17.620275
## final  value 17.620275 
## stopped after 100 iterations
## # weights:  19
## initial  value 286.140516 
## iter  10 value 67.968321
## iter  20 value 60.050114
## iter  30 value 57.704353
## iter  40 value 56.453216
## iter  50 value 51.086355
## iter  60 value 48.772503
## iter  70 value 48.745171
## iter  80 value 48.731062
## iter  90 value 48.698778
## iter 100 value 48.690023
## final  value 48.690023 
## stopped after 100 iterations
## # weights:  55
## initial  value 433.619397 
## iter  10 value 57.610825
## iter  20 value 42.786252
## iter  30 value 31.444265
## iter  40 value 27.802331
## iter  50 value 26.940740
## iter  60 value 26.813185
## iter  70 value 26.760696
## iter  80 value 26.743923
## iter  90 value 26.704289
## iter 100 value 26.695546
## final  value 26.695546 
## stopped after 100 iterations
## # weights:  91
## initial  value 618.754562 
## iter  10 value 51.020057
## iter  20 value 32.018658
## iter  30 value 22.114147
## iter  40 value 18.894098
## iter  50 value 18.075407
## iter  60 value 17.493080
## iter  70 value 16.880867
## iter  80 value 16.348205
## iter  90 value 15.680411
## iter 100 value 15.002180
## final  value 15.002180 
## stopped after 100 iterations
## # weights:  19
## initial  value 310.423260 
## iter  10 value 76.434280
## iter  20 value 68.106404
## iter  30 value 66.571045
## iter  40 value 66.007422
## iter  50 value 65.807506
## final  value 65.786419 
## converged
## # weights:  55
## initial  value 366.735350 
## iter  10 value 70.751352
## iter  20 value 57.511960
## iter  30 value 52.424740
## iter  40 value 51.445611
## iter  50 value 51.082055
## iter  60 value 51.027755
## iter  70 value 51.026924
## final  value 51.026923 
## converged
## # weights:  91
## initial  value 204.304308 
## iter  10 value 66.444035
## iter  20 value 56.818583
## iter  30 value 53.193102
## iter  40 value 51.737480
## iter  50 value 51.116146
## iter  60 value 50.350090
## iter  70 value 48.967710
## iter  80 value 47.713449
## iter  90 value 47.342505
## iter 100 value 47.323737
## final  value 47.323737 
## stopped after 100 iterations
## # weights:  19
## initial  value 322.152977 
## iter  10 value 65.660011
## iter  20 value 61.645001
## iter  30 value 57.316104
## iter  40 value 55.264432
## iter  50 value 51.508195
## iter  60 value 51.203824
## iter  70 value 51.188498
## iter  80 value 51.185598
## iter  90 value 51.185337
## final  value 51.185332 
## converged
## # weights:  55
## initial  value 357.452502 
## iter  10 value 61.083439
## iter  20 value 46.375558
## iter  30 value 41.763029
## iter  40 value 39.209491
## iter  50 value 38.572570
## iter  60 value 38.115320
## iter  70 value 37.876056
## iter  80 value 37.776982
## iter  90 value 37.510648
## iter 100 value 37.378144
## final  value 37.378144 
## stopped after 100 iterations
## # weights:  91
## initial  value 318.431606 
## iter  10 value 52.403096
## iter  20 value 35.680106
## iter  30 value 31.314054
## iter  40 value 29.741903
## iter  50 value 29.085445
## iter  60 value 28.730705
## iter  70 value 28.529878
## iter  80 value 28.473597
## iter  90 value 28.412492
## iter 100 value 28.361008
## final  value 28.361008 
## stopped after 100 iterations
## # weights:  19
## initial  value 320.808748 
## iter  10 value 72.856907
## iter  20 value 58.070407
## iter  30 value 55.653038
## iter  40 value 53.024776
## iter  50 value 53.022427
## iter  60 value 53.021015
## iter  70 value 53.020318
## final  value 53.019822 
## converged
## # weights:  55
## initial  value 324.528319 
## iter  10 value 62.537678
## iter  20 value 52.166501
## iter  30 value 44.559839
## iter  40 value 41.197042
## iter  50 value 39.674757
## iter  60 value 38.303472
## iter  70 value 36.677108
## iter  80 value 36.073153
## iter  90 value 34.812846
## iter 100 value 33.625456
## final  value 33.625456 
## stopped after 100 iterations
## # weights:  91
## initial  value 335.773907 
## iter  10 value 62.784522
## iter  20 value 38.266584
## iter  30 value 31.949258
## iter  40 value 26.171720
## iter  50 value 24.692719
## iter  60 value 23.578569
## iter  70 value 22.413985
## iter  80 value 21.573683
## iter  90 value 21.262099
## iter 100 value 21.049293
## final  value 21.049293 
## stopped after 100 iterations
## # weights:  19
## initial  value 454.888176 
## iter  10 value 143.168823
## iter  20 value 95.125947
## iter  30 value 70.421425
## iter  40 value 65.791573
## iter  50 value 65.477218
## iter  60 value 65.450502
## iter  60 value 65.450502
## iter  60 value 65.450502
## final  value 65.450502 
## converged
## # weights:  55
## initial  value 314.293858 
## iter  10 value 74.105026
## iter  20 value 59.284915
## iter  30 value 55.282031
## iter  40 value 53.925213
## iter  50 value 53.837061
## iter  60 value 53.792521
## final  value 53.789968 
## converged
## # weights:  91
## initial  value 263.706979 
## iter  10 value 67.160810
## iter  20 value 59.099429
## iter  30 value 56.351979
## iter  40 value 54.507413
## iter  50 value 52.568696
## iter  60 value 51.858068
## iter  70 value 50.047770
## iter  80 value 49.625949
## iter  90 value 49.549901
## iter 100 value 49.543079
## final  value 49.543079 
## stopped after 100 iterations
## # weights:  19
## initial  value 254.912422 
## iter  10 value 79.589087
## iter  20 value 71.400900
## iter  30 value 69.494194
## iter  40 value 67.641443
## iter  50 value 67.450935
## iter  60 value 66.842660
## iter  70 value 66.774068
## iter  80 value 66.745096
## iter  90 value 66.736458
## iter 100 value 66.732094
## final  value 66.732094 
## stopped after 100 iterations
## # weights:  55
## initial  value 337.913969 
## iter  10 value 57.289622
## iter  20 value 49.737109
## iter  30 value 41.948278
## iter  40 value 40.554765
## iter  50 value 39.495482
## iter  60 value 38.919231
## iter  70 value 38.799059
## iter  80 value 38.707654
## iter  90 value 38.625796
## iter 100 value 38.506632
## final  value 38.506632 
## stopped after 100 iterations
## # weights:  91
## initial  value 257.551167 
## iter  10 value 49.140909
## iter  20 value 28.834240
## iter  30 value 20.998800
## iter  40 value 19.054485
## iter  50 value 18.171724
## iter  60 value 17.761349
## iter  70 value 17.158932
## iter  80 value 16.857366
## iter  90 value 16.232216
## iter 100 value 15.531906
## final  value 15.531906 
## stopped after 100 iterations
## # weights:  19
## initial  value 296.894633 
## iter  10 value 72.145730
## iter  20 value 57.445170
## iter  30 value 53.597687
## iter  40 value 52.470145
## iter  50 value 52.263771
## iter  60 value 52.224858
## final  value 52.224551 
## converged
## # weights:  55
## initial  value 322.553240 
## iter  10 value 77.265297
## iter  20 value 49.427576
## iter  30 value 42.732137
## iter  40 value 40.321243
## iter  50 value 37.445246
## iter  60 value 36.073924
## iter  70 value 34.645827
## iter  80 value 34.326359
## iter  90 value 34.217720
## iter 100 value 34.210187
## final  value 34.210187 
## stopped after 100 iterations
## # weights:  91
## initial  value 233.977796 
## iter  10 value 51.028490
## iter  20 value 29.087640
## iter  30 value 18.362564
## iter  40 value 14.747256
## iter  50 value 13.645102
## iter  60 value 13.090165
## iter  70 value 12.740968
## iter  80 value 12.548769
## iter  90 value 12.481595
## iter 100 value 12.464205
## final  value 12.464205 
## stopped after 100 iterations
## # weights:  19
## initial  value 317.873283 
## iter  10 value 79.182446
## iter  20 value 64.277511
## iter  30 value 61.190881
## iter  40 value 60.986496
## iter  50 value 60.986217
## final  value 60.986213 
## converged
## # weights:  55
## initial  value 169.456833 
## iter  10 value 66.391087
## iter  20 value 55.922990
## iter  30 value 49.698025
## iter  40 value 47.693081
## iter  50 value 47.416724
## iter  60 value 47.346937
## iter  70 value 47.345345
## final  value 47.345342 
## converged
## # weights:  91
## initial  value 191.442627 
## iter  10 value 62.898916
## iter  20 value 51.916094
## iter  30 value 47.555118
## iter  40 value 45.593482
## iter  50 value 44.641927
## iter  60 value 43.521118
## iter  70 value 43.088196
## iter  80 value 42.999918
## iter  90 value 42.983079
## iter 100 value 42.982596
## final  value 42.982596 
## stopped after 100 iterations
## # weights:  19
## initial  value 206.721059 
## iter  10 value 61.447995
## iter  20 value 55.202361
## iter  30 value 53.405659
## iter  40 value 52.279468
## iter  50 value 52.268785
## iter  60 value 52.254770
## final  value 52.242586 
## converged
## # weights:  55
## initial  value 196.645256 
## iter  10 value 75.101253
## iter  20 value 40.426783
## iter  30 value 33.746053
## iter  40 value 32.291478
## iter  50 value 31.977401
## iter  60 value 31.841583
## iter  70 value 31.768137
## iter  80 value 31.709328
## iter  90 value 31.664937
## iter 100 value 31.635661
## final  value 31.635661 
## stopped after 100 iterations
## # weights:  91
## initial  value 315.302980 
## iter  10 value 86.538388
## iter  20 value 40.038333
## iter  30 value 23.670420
## iter  40 value 14.354256
## iter  50 value 9.419611
## iter  60 value 8.552835
## iter  70 value 8.504782
## iter  80 value 8.423477
## iter  90 value 8.377695
## iter 100 value 8.333636
## final  value 8.333636 
## stopped after 100 iterations
## # weights:  19
## initial  value 283.060022 
## iter  10 value 83.888841
## iter  20 value 51.176309
## iter  30 value 44.073401
## iter  40 value 39.690609
## iter  50 value 38.015091
## iter  60 value 35.706857
## iter  70 value 34.523620
## iter  80 value 34.345722
## iter  90 value 34.336098
## iter 100 value 34.329880
## final  value 34.329880 
## stopped after 100 iterations
## # weights:  55
## initial  value 284.801277 
## iter  10 value 57.317377
## iter  20 value 38.626163
## iter  30 value 28.417755
## iter  40 value 27.392488
## iter  50 value 26.958732
## iter  60 value 26.841348
## iter  70 value 26.763140
## iter  80 value 26.749483
## iter  90 value 26.719329
## iter 100 value 26.669842
## final  value 26.669842 
## stopped after 100 iterations
## # weights:  91
## initial  value 361.428730 
## iter  10 value 44.946753
## iter  20 value 29.243480
## iter  30 value 21.693530
## iter  40 value 19.139937
## iter  50 value 17.686367
## iter  60 value 17.203408
## iter  70 value 16.978446
## iter  80 value 16.299943
## iter  90 value 15.507121
## iter 100 value 13.740452
## final  value 13.740452 
## stopped after 100 iterations
## # weights:  19
## initial  value 252.767875 
## iter  10 value 92.193205
## iter  20 value 63.764333
## iter  30 value 56.956642
## iter  40 value 55.586281
## iter  50 value 55.380958
## final  value 55.367261 
## converged
## # weights:  55
## initial  value 308.685555 
## iter  10 value 62.803306
## iter  20 value 54.108569
## iter  30 value 51.288241
## iter  40 value 51.155347
## iter  50 value 50.953108
## iter  60 value 50.678570
## iter  70 value 50.441104
## iter  80 value 50.425281
## final  value 50.425275 
## converged
## # weights:  91
## initial  value 532.481240 
## iter  10 value 71.995509
## iter  20 value 51.777498
## iter  30 value 48.717216
## iter  40 value 46.639764
## iter  50 value 45.730042
## iter  60 value 43.630254
## iter  70 value 43.053210
## iter  80 value 43.036417
## iter  90 value 43.035198
## final  value 43.035191 
## converged
## # weights:  19
## initial  value 261.700757 
## iter  10 value 63.060222
## iter  20 value 46.934222
## iter  30 value 44.755980
## iter  40 value 42.708972
## iter  50 value 42.140828
## iter  60 value 41.628658
## iter  70 value 41.514030
## iter  80 value 41.491299
## iter  90 value 41.481058
## iter 100 value 41.476349
## final  value 41.476349 
## stopped after 100 iterations
## # weights:  55
## initial  value 413.121606 
## iter  10 value 61.780225
## iter  20 value 39.037916
## iter  30 value 33.858658
## iter  40 value 32.545975
## iter  50 value 31.188283
## iter  60 value 28.666246
## iter  70 value 28.326579
## iter  80 value 28.022920
## iter  90 value 27.891335
## iter 100 value 27.767959
## final  value 27.767959 
## stopped after 100 iterations
## # weights:  91
## initial  value 362.558467 
## iter  10 value 40.942861
## iter  20 value 26.951296
## iter  30 value 17.236100
## iter  40 value 14.268325
## iter  50 value 13.801461
## iter  60 value 13.526178
## iter  70 value 13.211304
## iter  80 value 12.907852
## iter  90 value 12.789617
## iter 100 value 12.731299
## final  value 12.731299 
## stopped after 100 iterations
## # weights:  19
## initial  value 262.645663 
## iter  10 value 83.499982
## iter  20 value 72.104354
## iter  30 value 67.078339
## iter  40 value 63.278018
## iter  50 value 61.930370
## iter  60 value 61.511909
## iter  70 value 60.839712
## iter  80 value 60.747543
## final  value 60.747037 
## converged
## # weights:  55
## initial  value 238.575648 
## iter  10 value 66.363015
## iter  20 value 53.677529
## iter  30 value 43.068350
## iter  40 value 38.368414
## iter  50 value 32.845501
## iter  60 value 29.051143
## iter  70 value 26.054765
## iter  80 value 25.125845
## iter  90 value 24.879074
## iter 100 value 24.797444
## final  value 24.797444 
## stopped after 100 iterations
## # weights:  91
## initial  value 221.937836 
## iter  10 value 78.882210
## iter  20 value 41.831953
## iter  30 value 34.140767
## iter  40 value 30.502203
## iter  50 value 29.660881
## iter  60 value 27.509184
## iter  70 value 25.477271
## iter  80 value 24.287569
## iter  90 value 22.475288
## iter 100 value 21.726551
## final  value 21.726551 
## stopped after 100 iterations
## # weights:  19
## initial  value 367.274281 
## iter  10 value 81.348105
## iter  20 value 68.967467
## iter  30 value 66.380004
## iter  40 value 66.197933
## final  value 66.197208 
## converged
## # weights:  55
## initial  value 387.095429 
## iter  10 value 66.194703
## iter  20 value 54.885813
## iter  30 value 53.155746
## iter  40 value 52.923618
## iter  50 value 52.837270
## iter  60 value 52.836926
## final  value 52.836926 
## converged
## # weights:  91
## initial  value 300.234581 
## iter  10 value 66.378804
## iter  20 value 55.312231
## iter  30 value 52.947860
## iter  40 value 51.981248
## iter  50 value 50.133835
## iter  60 value 49.282931
## iter  70 value 48.341092
## iter  80 value 48.245254
## iter  90 value 48.239979
## final  value 48.239952 
## converged
## # weights:  19
## initial  value 245.307419 
## iter  10 value 73.807998
## iter  20 value 60.289685
## iter  30 value 58.802311
## iter  40 value 56.827050
## iter  50 value 56.638554
## iter  60 value 56.377202
## iter  70 value 56.254515
## iter  80 value 56.223147
## iter  90 value 56.209954
## iter 100 value 56.198503
## final  value 56.198503 
## stopped after 100 iterations
## # weights:  55
## initial  value 155.885606 
## iter  10 value 59.440142
## iter  20 value 43.045686
## iter  30 value 39.924918
## iter  40 value 38.255238
## iter  50 value 38.029980
## iter  60 value 37.448862
## iter  70 value 37.359854
## iter  80 value 37.300948
## iter  90 value 36.913224
## iter 100 value 36.816669
## final  value 36.816669 
## stopped after 100 iterations
## # weights:  91
## initial  value 252.535339 
## iter  10 value 77.874456
## iter  20 value 51.870120
## iter  30 value 44.533985
## iter  40 value 41.447902
## iter  50 value 40.068225
## iter  60 value 37.059015
## iter  70 value 33.424977
## iter  80 value 31.647974
## iter  90 value 29.775956
## iter 100 value 26.832529
## final  value 26.832529 
## stopped after 100 iterations
## # weights:  19
## initial  value 315.535566 
## iter  10 value 69.367059
## iter  20 value 50.490764
## iter  30 value 46.129352
## iter  40 value 44.730976
## final  value 44.727671 
## converged
## # weights:  55
## initial  value 391.353348 
## iter  10 value 44.906271
## iter  20 value 28.799805
## iter  30 value 24.655788
## iter  40 value 23.691555
## iter  50 value 23.030399
## iter  60 value 22.185481
## iter  70 value 21.346111
## iter  80 value 21.100584
## iter  90 value 21.086203
## iter 100 value 21.084832
## final  value 21.084832 
## stopped after 100 iterations
## # weights:  91
## initial  value 202.513592 
## iter  10 value 49.174985
## iter  20 value 30.116655
## iter  30 value 18.681129
## iter  40 value 12.261379
## iter  50 value 9.599056
## iter  60 value 8.914015
## iter  70 value 8.876108
## iter  80 value 8.872280
## iter  90 value 8.871567
## iter 100 value 8.871003
## final  value 8.871003 
## stopped after 100 iterations
## # weights:  19
## initial  value 329.935941 
## iter  10 value 77.805107
## iter  20 value 63.538759
## iter  30 value 60.551581
## iter  40 value 60.076040
## iter  50 value 60.075805
## final  value 60.075798 
## converged
## # weights:  55
## initial  value 230.044660 
## iter  10 value 68.775586
## iter  20 value 55.704319
## iter  30 value 50.997603
## iter  40 value 50.328240
## iter  50 value 50.135247
## iter  60 value 50.129211
## final  value 50.129144 
## converged
## # weights:  91
## initial  value 293.915867 
## iter  10 value 63.326628
## iter  20 value 52.409662
## iter  30 value 47.545945
## iter  40 value 44.279315
## iter  50 value 43.448033
## iter  60 value 43.045363
## iter  70 value 42.918779
## iter  80 value 42.869175
## iter  90 value 42.868394
## iter 100 value 42.854086
## final  value 42.854086 
## stopped after 100 iterations
## # weights:  19
## initial  value 264.894498 
## iter  10 value 63.496258
## iter  20 value 51.972042
## iter  30 value 50.893540
## iter  40 value 48.523309
## iter  50 value 48.382398
## iter  60 value 48.327091
## iter  70 value 48.305216
## iter  80 value 48.296451
## iter  90 value 48.293194
## iter 100 value 48.288001
## final  value 48.288001 
## stopped after 100 iterations
## # weights:  55
## initial  value 427.545568 
## iter  10 value 51.952735
## iter  20 value 44.846605
## iter  30 value 37.628351
## iter  40 value 36.210680
## iter  50 value 35.289304
## iter  60 value 34.709799
## iter  70 value 34.493352
## iter  80 value 34.294622
## iter  90 value 34.219747
## iter 100 value 34.159603
## final  value 34.159603 
## stopped after 100 iterations
## # weights:  91
## initial  value 176.480150 
## iter  10 value 46.377184
## iter  20 value 30.528825
## iter  30 value 23.697477
## iter  40 value 21.179646
## iter  50 value 20.747525
## iter  60 value 20.456093
## iter  70 value 20.212684
## iter  80 value 20.117760
## iter  90 value 20.050517
## iter 100 value 20.018014
## final  value 20.018014 
## stopped after 100 iterations
## # weights:  55
## initial  value 270.015273 
## iter  10 value 69.506896
## iter  20 value 61.745338
## iter  30 value 59.802579
## iter  40 value 59.127278
## iter  50 value 58.962751
## iter  60 value 58.950862
## final  value 58.950846 
## converged

plot(modell_nn4)

Das beste Modell ergibt sich bei einem Hiddenlayer und einem decay weight von 0.1. So wurde eine Trainingsaccuracy von 93% erreicht. Auffallend ist zudem, dass bei einem decay weight von 0.1, es gar keine Rolle spielt, wie viel Hiddenlayer es gibt, da die Accuracy stets bei ca. 93% liegt. Die Modelle mit nur leicht veränderten weights, schließen etwas schlechter ab, mit einer Trainingsaccuracy von knapp über 89% bis 92%.

modell_nn4_best <- modell_nn4$bestTune
modell_nn4_best

##   size decay
## 6    3   0.1

Das beste Modell entsteht mit 5 Hidden Units und einem Weights decay von 0,1.

predict_testNN_4 = predict(modell_nn4, data_test)
#predict_testNN_4 <-sapply(predict_testNN_4,round,digits=0)
nn_table4 <- table(data_test$target, predict_testNN_4)

Auch auf den Testdaten perfomt das NN mit der Caret Funktion besser als die beiden Modelle mit der nnet-Funktion. Die Testaccuracy liegt bei über 91 %, der Recall bei 60%. Aber es wurden 5 Patienten fälschlicherweise als gesund gemeldet, obwohl Sie mit Corona infiziert sind. Dies wollen wir unebdingt vermeiden.

results_nn4 <- data.frame(actual = data_test$target, prediction = predict_testNN_4)
conf_nn4 <- confusionMatrix(nn_table4)
conf_nn4

## Confusion Matrix and Statistics
## 
##    predict_testNN_4
##      0  1
##   0 90  4
##   1  5  6
##                                           
##                Accuracy : 0.9143          
##                  95% CI : (0.8435, 0.9601)
##     No Information Rate : 0.9048          
##     P-Value [Acc > NIR] : 0.4516          
##                                           
##                   Kappa : 0.5239          
##                                           
##  Mcnemar's Test P-Value : 1.0000          
##                                           
##             Sensitivity : 0.9474          
##             Specificity : 0.6000          
##          Pos Pred Value : 0.9574          
##          Neg Pred Value : 0.5455          
##              Prevalence : 0.9048          
##          Detection Rate : 0.8571          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.7737          
##                                           
##        'Positive' Class : 0               
## 

acc_nn4 <- conf_nn4$overall[1]
sens_nn4 <- conf_nn4$byClass[1]
spec_nn4 <- conf_nn4$byClass[2]

Nun trainieren wir noch ein weiteres Neuronales Netz mit der Train Function, nun werden wir als Trainingsdatensatz aber einen upsampleten Datensatz, mit balancierter Responsevariable verwenden und die Daten zu vor Dummy Encodieren. Wir sind gespannt, wie es sich im Vergleich zum Modell mit dem “normalen” Trainingsdaten verhält.

set.seed(1910837262)
up_trainset_nn <- upSample(x = data_train[, -ncol(data_train)],
                     y = as.factor(data_train$target))                         
table(up_trainset_nn$target) 

## 
##   0   1 
## 380 380

up_trainset_nn <- up_trainset_nn %>%
    select(-Class)

for (f in ohe_feats){
  df_all_dummy = acm.disjonctif(up_trainset_nn[f])
  up_trainset_nn[f] = NULL
  up_trainset_nn = cbind(up_trainset_nn, df_all_dummy)
}

testset_nn <- data_test
for (f in ohe_feats){
  df_all_dummy = acm.disjonctif(testset_nn[f])
  testset_nn[f] = NULL
  testset_nn = cbind(testset_nn, df_all_dummy)
}

#Caret Modell mit upsamplet Trainingdsatensatz:
modell_nn5 <- train(up_trainset_nn[,-2], up_trainset_nn$target,
                  method = "nnet",
                  trControl= TrainingParameters_nn,
                  preProcess=c("scale","center")
)

## # weights:  22
## initial  value 493.250696 
## iter  10 value 261.647044
## iter  20 value 207.231245
## iter  30 value 201.548351
## iter  40 value 199.725889
## iter  50 value 199.555556
## iter  60 value 199.549818
## iter  70 value 199.539122
## iter  80 value 199.535649
## final  value 199.535645 
## converged
## # weights:  64
## initial  value 494.655937 
## iter  10 value 185.923247
## iter  20 value 157.355340
## iter  30 value 142.044914
## iter  40 value 132.734099
## iter  50 value 130.063650
## iter  60 value 130.008440
## final  value 130.008371 
## converged
## # weights:  106
## initial  value 618.232939 
## iter  10 value 186.273563
## iter  20 value 126.151469
## iter  30 value 109.831653
## iter  40 value 104.990130
## iter  50 value 104.007064
## iter  60 value 104.000064
## iter  70 value 103.999040
## final  value 103.998927 
## converged
## # weights:  22
## initial  value 486.101694 
## iter  10 value 271.907204
## iter  20 value 216.205880
## iter  30 value 207.213205
## iter  40 value 206.736992
## final  value 206.736292 
## converged
## # weights:  64
## initial  value 477.362750 
## iter  10 value 220.828987
## iter  20 value 207.362127
## iter  30 value 180.154796
## iter  40 value 167.337717
## iter  50 value 154.062601
## iter  60 value 133.102227
## iter  70 value 129.891245
## iter  80 value 127.371284
## iter  90 value 126.864525
## iter 100 value 126.731737
## final  value 126.731737 
## stopped after 100 iterations
## # weights:  106
## initial  value 521.986902 
## iter  10 value 243.731071
## iter  20 value 183.154071
## iter  30 value 155.876847
## iter  40 value 141.660653
## iter  50 value 134.570960
## iter  60 value 126.757897
## iter  70 value 121.606665
## iter  80 value 119.674971
## iter  90 value 118.831436
## iter 100 value 118.344860
## final  value 118.344860 
## stopped after 100 iterations
## # weights:  22
## initial  value 499.813766 
## iter  10 value 269.859944
## iter  20 value 205.227041
## iter  30 value 197.208419
## iter  40 value 196.054608
## iter  50 value 193.359043
## iter  60 value 192.531550
## iter  70 value 192.469719
## iter  80 value 192.454492
## iter  90 value 192.450671
## iter 100 value 192.450460
## final  value 192.450460 
## stopped after 100 iterations
## # weights:  64
## initial  value 537.491108 
## iter  10 value 181.376563
## iter  20 value 147.707208
## iter  30 value 125.790749
## iter  40 value 118.360951
## iter  50 value 115.975205
## iter  60 value 113.136751
## iter  70 value 110.788224
## iter  80 value 109.411034
## iter  90 value 108.965255
## iter 100 value 108.593799
## final  value 108.593799 
## stopped after 100 iterations
## # weights:  106
## initial  value 568.808717 
## iter  10 value 194.498413
## iter  20 value 117.293496
## iter  30 value 90.964483
## iter  40 value 78.694259
## iter  50 value 67.814226
## iter  60 value 65.341068
## iter  70 value 63.667616
## iter  80 value 62.021714
## iter  90 value 61.319742
## iter 100 value 60.815903
## final  value 60.815903 
## stopped after 100 iterations
## # weights:  22
## initial  value 506.076053 
## iter  10 value 293.458468
## iter  20 value 239.281743
## iter  30 value 221.098826
## iter  40 value 220.209462
## iter  50 value 218.996601
## iter  60 value 218.505202
## iter  70 value 217.681099
## iter  80 value 217.261147
## final  value 217.257721 
## converged
## # weights:  64
## initial  value 479.966921 
## iter  10 value 187.112629
## iter  20 value 146.611734
## iter  30 value 119.928953
## iter  40 value 111.850914
## iter  50 value 108.439536
## iter  60 value 107.461300
## iter  70 value 107.353511
## iter  80 value 107.350098
## iter  90 value 107.349921
## final  value 107.349904 
## converged
## # weights:  106
## initial  value 487.285719 
## iter  10 value 194.230833
## iter  20 value 129.810540
## iter  30 value 97.896225
## iter  40 value 78.718940
## iter  50 value 66.894285
## iter  60 value 56.322598
## iter  70 value 49.063578
## iter  80 value 47.878884
## iter  90 value 46.970191
## iter 100 value 46.718822
## final  value 46.718822 
## stopped after 100 iterations
## # weights:  22
## initial  value 497.015109 
## iter  10 value 267.456264
## iter  20 value 230.718261
## iter  30 value 224.832562
## iter  40 value 223.566927
## final  value 223.556966 
## converged
## # weights:  64
## initial  value 464.198157 
## iter  10 value 202.596767
## iter  20 value 183.912449
## iter  30 value 172.839912
## iter  40 value 169.578847
## iter  50 value 166.610716
## iter  60 value 165.237420
## iter  70 value 164.123359
## iter  80 value 164.075438
## final  value 164.075264 
## converged
## # weights:  106
## initial  value 522.195768 
## iter  10 value 232.716206
## iter  20 value 189.309805
## iter  30 value 149.988535
## iter  40 value 133.252242
## iter  50 value 125.150684
## iter  60 value 121.103424
## iter  70 value 117.811037
## iter  80 value 115.985359
## iter  90 value 115.132381
## iter 100 value 114.647231
## final  value 114.647231 
## stopped after 100 iterations
## # weights:  22
## initial  value 487.935365 
## iter  10 value 261.933517
## iter  20 value 220.855645
## iter  30 value 214.960230
## iter  40 value 209.865108
## iter  50 value 209.299737
## iter  60 value 209.009950
## iter  70 value 207.533366
## iter  80 value 207.356557
## iter  90 value 207.295256
## iter 100 value 207.204601
## final  value 207.204601 
## stopped after 100 iterations
## # weights:  64
## initial  value 512.616609 
## iter  10 value 212.073007
## iter  20 value 161.846817
## iter  30 value 142.740344
## iter  40 value 135.468559
## iter  50 value 128.934725
## iter  60 value 127.611404
## iter  70 value 127.254486
## iter  80 value 126.492869
## iter  90 value 125.935272
## iter 100 value 125.517915
## final  value 125.517915 
## stopped after 100 iterations
## # weights:  106
## initial  value 489.190572 
## iter  10 value 195.239424
## iter  20 value 140.770053
## iter  30 value 98.235881
## iter  40 value 80.820329
## iter  50 value 77.898583
## iter  60 value 75.335549
## iter  70 value 74.899541
## iter  80 value 74.624690
## iter  90 value 69.556765
## iter 100 value 67.327042
## final  value 67.327042 
## stopped after 100 iterations
## # weights:  22
## initial  value 500.894604 
## iter  10 value 239.074952
## iter  20 value 207.098435
## iter  30 value 203.945194
## iter  40 value 203.699530
## iter  50 value 203.681553
## final  value 203.681504 
## converged
## # weights:  64
## initial  value 484.299959 
## iter  10 value 188.475848
## iter  20 value 149.539878
## iter  30 value 129.154516
## iter  40 value 122.240683
## iter  50 value 120.138977
## iter  60 value 119.860344
## iter  70 value 119.771342
## iter  80 value 119.604807
## iter  90 value 119.600916
## final  value 119.600878 
## converged
## # weights:  106
## initial  value 521.003756 
## iter  10 value 187.711872
## iter  20 value 136.719411
## iter  30 value 114.330956
## iter  40 value 103.521370
## iter  50 value 94.829828
## iter  60 value 92.042201
## iter  70 value 91.459898
## iter  80 value 91.136035
## iter  90 value 90.956600
## iter 100 value 90.860097
## final  value 90.860097 
## stopped after 100 iterations
## # weights:  22
## initial  value 484.762682 
## iter  10 value 241.663402
## iter  20 value 228.631823
## iter  30 value 216.569097
## iter  40 value 210.495670
## iter  50 value 210.343248
## iter  60 value 210.342421
## final  value 210.342359 
## converged
## # weights:  64
## initial  value 474.932715 
## iter  10 value 268.527652
## iter  20 value 195.727091
## iter  30 value 176.478247
## iter  40 value 163.286264
## iter  50 value 158.020244
## iter  60 value 155.521383
## iter  70 value 154.302528
## iter  80 value 154.144604
## iter  90 value 154.140696
## iter 100 value 154.132262
## final  value 154.132262 
## stopped after 100 iterations
## # weights:  106
## initial  value 493.287996 
## iter  10 value 198.261732
## iter  20 value 159.700190
## iter  30 value 140.841161
## iter  40 value 130.112202
## iter  50 value 124.472536
## iter  60 value 122.496786
## iter  70 value 120.861351
## iter  80 value 118.394076
## iter  90 value 114.233775
## iter 100 value 112.762639
## final  value 112.762639 
## stopped after 100 iterations
## # weights:  22
## initial  value 481.384152 
## iter  10 value 276.034183
## iter  20 value 213.926075
## iter  30 value 204.676772
## iter  40 value 204.556915
## iter  50 value 204.517481
## iter  60 value 204.501071
## iter  70 value 204.480030
## iter  70 value 204.480028
## iter  70 value 204.480028
## final  value 204.480028 
## converged
## # weights:  64
## initial  value 485.773315 
## iter  10 value 182.345405
## iter  20 value 138.085683
## iter  30 value 122.195990
## iter  40 value 116.137478
## iter  50 value 115.381193
## iter  60 value 115.068872
## iter  70 value 114.868972
## iter  80 value 114.684889
## iter  90 value 114.405134
## iter 100 value 114.291189
## final  value 114.291189 
## stopped after 100 iterations
## # weights:  106
## initial  value 599.135359 
## iter  10 value 173.622545
## iter  20 value 111.713362
## iter  30 value 78.432991
## iter  40 value 67.596249
## iter  50 value 60.777746
## iter  60 value 57.620806
## iter  70 value 55.846580
## iter  80 value 55.459143
## iter  90 value 55.220997
## iter 100 value 55.065950
## final  value 55.065950 
## stopped after 100 iterations
## # weights:  22
## initial  value 527.826128 
## iter  10 value 283.540536
## iter  20 value 229.137257
## iter  30 value 222.464152
## iter  40 value 214.759629
## iter  50 value 214.071233
## iter  60 value 214.067850
## iter  70 value 214.065466
## iter  80 value 214.065079
## final  value 214.064749 
## converged
## # weights:  64
## initial  value 475.814788 
## iter  10 value 224.880875
## iter  20 value 177.458752
## iter  30 value 156.434289
## iter  40 value 139.781924
## iter  50 value 135.685051
## iter  60 value 129.778596
## iter  70 value 126.762204
## iter  80 value 126.543241
## iter  90 value 126.452245
## iter 100 value 126.444789
## final  value 126.444789 
## stopped after 100 iterations
## # weights:  106
## initial  value 488.218610 
## iter  10 value 199.001926
## iter  20 value 158.770537
## iter  30 value 108.911611
## iter  40 value 90.648199
## iter  50 value 65.648496
## iter  60 value 50.673020
## iter  70 value 44.898738
## iter  80 value 41.177145
## iter  90 value 38.335727
## iter 100 value 36.212441
## final  value 36.212441 
## stopped after 100 iterations
## # weights:  22
## initial  value 472.079367 
## iter  10 value 293.639905
## iter  20 value 236.754359
## iter  30 value 221.819838
## iter  40 value 219.992009
## final  value 219.980018 
## converged
## # weights:  64
## initial  value 553.360171 
## iter  10 value 204.133295
## iter  20 value 171.660669
## iter  30 value 163.624920
## iter  40 value 158.689244
## iter  50 value 156.287390
## iter  60 value 155.175508
## iter  70 value 155.076551
## iter  80 value 155.070223
## final  value 155.070169 
## converged
## # weights:  106
## initial  value 502.942670 
## iter  10 value 199.341727
## iter  20 value 157.916575
## iter  30 value 140.815669
## iter  40 value 133.661660
## iter  50 value 129.344405
## iter  60 value 125.452794
## iter  70 value 124.352318
## iter  80 value 123.396002
## iter  90 value 123.175094
## iter 100 value 123.152156
## final  value 123.152156 
## stopped after 100 iterations
## # weights:  22
## initial  value 457.622360 
## iter  10 value 290.283561
## iter  20 value 220.197162
## iter  30 value 208.640633
## iter  40 value 204.713790
## iter  50 value 202.470318
## iter  60 value 202.202921
## iter  70 value 202.187062
## iter  80 value 202.180533
## iter  90 value 202.173077
## iter 100 value 202.171962
## final  value 202.171962 
## stopped after 100 iterations
## # weights:  64
## initial  value 478.119361 
## iter  10 value 201.199462
## iter  20 value 159.690269
## iter  30 value 141.041601
## iter  40 value 127.661395
## iter  50 value 124.593887
## iter  60 value 122.936259
## iter  70 value 122.373508
## iter  80 value 122.317488
## iter  90 value 122.258359
## iter 100 value 122.200667
## final  value 122.200667 
## stopped after 100 iterations
## # weights:  106
## initial  value 495.168867 
## iter  10 value 184.374813
## iter  20 value 119.940050
## iter  30 value 81.458157
## iter  40 value 56.602053
## iter  50 value 51.182952
## iter  60 value 45.736550
## iter  70 value 43.600108
## iter  80 value 42.468799
## iter  90 value 42.035533
## iter 100 value 41.162377
## final  value 41.162377 
## stopped after 100 iterations
## # weights:  22
## initial  value 502.667674 
## iter  10 value 208.836573
## iter  20 value 200.346606
## iter  30 value 195.939692
## iter  40 value 192.279084
## iter  50 value 189.778232
## iter  60 value 189.766882
## final  value 189.766802 
## converged
## # weights:  64
## initial  value 493.030021 
## iter  10 value 223.886905
## iter  20 value 174.302831
## iter  30 value 137.839612
## iter  40 value 122.193055
## iter  50 value 119.615738
## iter  60 value 117.142941
## iter  70 value 115.400674
## iter  80 value 112.659130
## iter  90 value 110.151726
## iter 100 value 109.781726
## final  value 109.781726 
## stopped after 100 iterations
## # weights:  106
## initial  value 445.026036 
## iter  10 value 170.823584
## iter  20 value 125.361823
## iter  30 value 101.785923
## iter  40 value 89.514944
## iter  50 value 78.846046
## iter  60 value 74.043250
## iter  70 value 72.677529
## iter  80 value 72.609824
## iter  90 value 72.607706
## final  value 72.607697 
## converged
## # weights:  22
## initial  value 474.443366 
## iter  10 value 214.744781
## iter  20 value 207.935595
## iter  30 value 207.636157
## final  value 207.636129 
## converged
## # weights:  64
## initial  value 461.706822 
## iter  10 value 202.739386
## iter  20 value 176.048635
## iter  30 value 171.355899
## iter  40 value 169.507998
## iter  50 value 168.142605
## iter  60 value 167.985764
## iter  70 value 167.976147
## final  value 167.976102 
## converged
## # weights:  106
## initial  value 546.126032 
## iter  10 value 188.562222
## iter  20 value 150.666506
## iter  30 value 129.026099
## iter  40 value 117.601497
## iter  50 value 113.181931
## iter  60 value 110.486812
## iter  70 value 107.464842
## iter  80 value 105.054630
## iter  90 value 103.692453
## iter 100 value 103.412434
## final  value 103.412434 
## stopped after 100 iterations
## # weights:  22
## initial  value 455.501277 
## iter  10 value 214.191773
## iter  20 value 205.879202
## iter  30 value 202.860037
## iter  40 value 200.613854
## iter  50 value 200.573906
## iter  60 value 200.572468
## iter  70 value 200.567618
## final  value 200.567607 
## converged
## # weights:  64
## initial  value 433.546062 
## iter  10 value 251.499049
## iter  20 value 217.810536
## iter  30 value 190.891446
## iter  40 value 168.629177
## iter  50 value 148.744569
## iter  60 value 131.583181
## iter  70 value 129.056732
## iter  80 value 126.735649
## iter  90 value 123.894470
## iter 100 value 122.780144
## final  value 122.780144 
## stopped after 100 iterations
## # weights:  106
## initial  value 460.349657 
## iter  10 value 178.240460
## iter  20 value 121.886572
## iter  30 value 108.232740
## iter  40 value 98.167257
## iter  50 value 92.193318
## iter  60 value 90.813202
## iter  70 value 90.326533
## iter  80 value 90.138843
## iter  90 value 89.854712
## iter 100 value 89.258725
## final  value 89.258725 
## stopped after 100 iterations
## # weights:  22
## initial  value 497.942852 
## iter  10 value 244.483997
## iter  20 value 212.801715
## iter  30 value 208.296937
## iter  40 value 205.479980
## iter  50 value 199.011198
## iter  60 value 198.947638
## iter  70 value 198.925121
## iter  80 value 198.894470
## iter  90 value 198.885483
## final  value 198.884950 
## converged
## # weights:  64
## initial  value 492.347645 
## iter  10 value 207.238783
## iter  20 value 172.708162
## iter  30 value 143.748804
## iter  40 value 124.541473
## iter  50 value 101.405032
## iter  60 value 96.063187
## iter  70 value 95.144925
## iter  80 value 94.706162
## iter  90 value 94.393538
## iter 100 value 94.160725
## final  value 94.160725 
## stopped after 100 iterations
## # weights:  106
## initial  value 503.662331 
## iter  10 value 189.491363
## iter  20 value 135.217579
## iter  30 value 114.382465
## iter  40 value 106.696559
## iter  50 value 104.652508
## iter  60 value 103.186564
## iter  70 value 99.278303
## iter  80 value 95.367204
## iter  90 value 94.821899
## iter 100 value 93.986369
## final  value 93.986369 
## stopped after 100 iterations
## # weights:  22
## initial  value 488.271407 
## iter  10 value 251.244646
## iter  20 value 221.361642
## iter  30 value 219.451367
## final  value 219.421050 
## converged
## # weights:  64
## initial  value 482.849339 
## iter  10 value 195.287369
## iter  20 value 171.591981
## iter  30 value 163.208384
## iter  40 value 156.920422
## iter  50 value 148.589871
## iter  60 value 141.005759
## iter  70 value 138.507873
## iter  80 value 137.562405
## iter  90 value 137.550029
## final  value 137.549567 
## converged
## # weights:  106
## initial  value 508.964363 
## iter  10 value 197.221760
## iter  20 value 153.391191
## iter  30 value 140.874922
## iter  40 value 133.269329
## iter  50 value 129.309723
## iter  60 value 127.594878
## iter  70 value 127.204548
## iter  80 value 127.061611
## iter  90 value 127.001774
## iter 100 value 126.945202
## final  value 126.945202 
## stopped after 100 iterations
## # weights:  22
## initial  value 475.495529 
## iter  10 value 300.105265
## iter  20 value 247.171828
## iter  30 value 236.613335
## iter  40 value 223.463220
## iter  50 value 221.129310
## iter  60 value 218.474020
## iter  70 value 218.148818
## final  value 218.148738 
## converged
## # weights:  64
## initial  value 483.071397 
## iter  10 value 192.037804
## iter  20 value 169.939711
## iter  30 value 139.849598
## iter  40 value 124.938579
## iter  50 value 121.608358
## iter  60 value 119.603849
## iter  70 value 118.164425
## iter  80 value 115.816512
## iter  90 value 114.870099
## iter 100 value 114.614717
## final  value 114.614717 
## stopped after 100 iterations
## # weights:  106
## initial  value 513.679373 
## iter  10 value 207.620273
## iter  20 value 159.986945
## iter  30 value 100.541248
## iter  40 value 85.271240
## iter  50 value 76.363491
## iter  60 value 72.688590
## iter  70 value 72.073593
## iter  80 value 71.331773
## iter  90 value 69.754310
## iter 100 value 69.152381
## final  value 69.152381 
## stopped after 100 iterations
## # weights:  22
## initial  value 482.601259 
## iter  10 value 262.725097
## iter  20 value 221.831658
## iter  30 value 208.543236
## iter  40 value 208.423240
## iter  50 value 208.333408
## iter  60 value 208.329510
## iter  70 value 208.314749
## final  value 208.314302 
## converged
## # weights:  64
## initial  value 520.801090 
## iter  10 value 197.795988
## iter  20 value 160.290480
## iter  30 value 149.241291
## iter  40 value 139.346923
## iter  50 value 127.721738
## iter  60 value 116.873078
## iter  70 value 110.601892
## iter  80 value 106.807490
## iter  90 value 103.715181
## iter 100 value 102.575314
## final  value 102.575314 
## stopped after 100 iterations
## # weights:  106
## initial  value 504.017632 
## iter  10 value 225.293592
## iter  20 value 142.427804
## iter  30 value 126.218868
## iter  40 value 119.760895
## iter  50 value 112.271422
## iter  60 value 109.481650
## iter  70 value 105.520666
## iter  80 value 103.833757
## iter  90 value 102.931339
## iter 100 value 101.673253
## final  value 101.673253 
## stopped after 100 iterations
## # weights:  22
## initial  value 472.914877 
## iter  10 value 260.866747
## iter  20 value 227.936098
## iter  30 value 226.148098
## final  value 226.142932 
## converged
## # weights:  64
## initial  value 493.717486 
## iter  10 value 235.129177
## iter  20 value 201.854245
## iter  30 value 174.342030
## iter  40 value 163.690875
## iter  50 value 158.405138
## iter  60 value 157.067754
## iter  70 value 156.755708
## iter  80 value 156.752223
## final  value 156.752197 
## converged
## # weights:  106
## initial  value 462.132473 
## iter  10 value 203.614554
## iter  20 value 168.506311
## iter  30 value 149.275431
## iter  40 value 136.741911
## iter  50 value 132.865040
## iter  60 value 127.046393
## iter  70 value 125.222114
## iter  80 value 123.390444
## iter  90 value 121.768152
## iter 100 value 120.350631
## final  value 120.350631 
## stopped after 100 iterations
## # weights:  22
## initial  value 477.725045 
## iter  10 value 332.323953
## iter  20 value 238.416160
## iter  30 value 226.065089
## iter  40 value 223.371333
## iter  50 value 223.239602
## iter  60 value 223.186187
## iter  70 value 223.176851
## iter  80 value 223.174169
## iter  90 value 223.173527
## iter 100 value 223.171580
## final  value 223.171580 
## stopped after 100 iterations
## # weights:  64
## initial  value 544.726372 
## iter  10 value 216.883726
## iter  20 value 178.906676
## iter  30 value 146.708566
## iter  40 value 137.261898
## iter  50 value 131.654277
## iter  60 value 127.755316
## iter  70 value 122.413199
## iter  80 value 121.810442
## iter  90 value 121.664216
## iter 100 value 121.525437
## final  value 121.525437 
## stopped after 100 iterations
## # weights:  106
## initial  value 524.890677 
## iter  10 value 230.736882
## iter  20 value 154.815111
## iter  30 value 127.323943
## iter  40 value 104.619244
## iter  50 value 99.165184
## iter  60 value 87.188356
## iter  70 value 74.255085
## iter  80 value 63.942511
## iter  90 value 58.371071
## iter 100 value 57.238133
## final  value 57.238133 
## stopped after 100 iterations
## # weights:  22
## initial  value 517.842480 
## iter  10 value 213.959501
## iter  20 value 206.566352
## iter  30 value 200.328303
## iter  40 value 191.476410
## iter  50 value 191.162000
## iter  60 value 191.160287
## iter  70 value 191.159575
## iter  80 value 191.159066
## iter  90 value 191.158071
## final  value 191.158057 
## converged
## # weights:  64
## initial  value 477.257590 
## iter  10 value 191.062763
## iter  20 value 156.775064
## iter  30 value 107.890187
## iter  40 value 99.501775
## iter  50 value 97.399875
## iter  60 value 96.507247
## iter  70 value 96.248127
## iter  80 value 96.159021
## iter  90 value 96.147748
## iter 100 value 96.143609
## final  value 96.143609 
## stopped after 100 iterations
## # weights:  106
## initial  value 474.932286 
## iter  10 value 171.410968
## iter  20 value 103.123374
## iter  30 value 69.846476
## iter  40 value 53.590904
## iter  50 value 48.727833
## iter  60 value 48.332382
## iter  70 value 48.252608
## iter  80 value 47.642065
## iter  90 value 47.133472
## iter 100 value 46.888295
## final  value 46.888295 
## stopped after 100 iterations
## # weights:  22
## initial  value 493.985557 
## iter  10 value 234.114006
## iter  20 value 220.200961
## iter  30 value 218.574913
## final  value 218.572405 
## converged
## # weights:  64
## initial  value 479.998719 
## iter  10 value 277.874410
## iter  20 value 214.577563
## iter  30 value 173.801014
## iter  40 value 164.242906
## iter  50 value 161.792829
## iter  60 value 160.917004
## iter  70 value 160.190197
## iter  80 value 143.830896
## iter  90 value 142.978101
## iter 100 value 142.896483
## final  value 142.896483 
## stopped after 100 iterations
## # weights:  106
## initial  value 519.619080 
## iter  10 value 189.553982
## iter  20 value 152.406895
## iter  30 value 139.535914
## iter  40 value 131.200443
## iter  50 value 126.416309
## iter  60 value 124.392750
## iter  70 value 123.297123
## iter  80 value 122.777366
## iter  90 value 122.481699
## iter 100 value 122.329251
## final  value 122.329251 
## stopped after 100 iterations
## # weights:  22
## initial  value 499.358501 
## iter  10 value 230.032289
## iter  20 value 206.556718
## iter  30 value 200.207924
## iter  40 value 196.646833
## iter  50 value 192.805068
## iter  60 value 192.741924
## iter  70 value 192.706983
## iter  80 value 192.665183
## iter  90 value 192.599258
## iter 100 value 192.589877
## final  value 192.589877 
## stopped after 100 iterations
## # weights:  64
## initial  value 493.996606 
## iter  10 value 189.959799
## iter  20 value 132.004498
## iter  30 value 116.457401
## iter  40 value 110.362993
## iter  50 value 108.821375
## iter  60 value 108.629210
## iter  70 value 108.399098
## iter  80 value 108.302242
## iter  90 value 107.130601
## iter 100 value 106.769495
## final  value 106.769495 
## stopped after 100 iterations
## # weights:  106
## initial  value 474.732801 
## iter  10 value 184.710814
## iter  20 value 134.934449
## iter  30 value 96.853212
## iter  40 value 63.733234
## iter  50 value 51.462917
## iter  60 value 48.506150
## iter  70 value 47.961837
## iter  80 value 46.046555
## iter  90 value 45.455620
## iter 100 value 45.325008
## final  value 45.325008 
## stopped after 100 iterations
## # weights:  22
## initial  value 534.273849 
## iter  10 value 230.563112
## iter  20 value 211.833016
## iter  30 value 209.455443
## iter  40 value 209.072361
## final  value 209.072300 
## converged
## # weights:  64
## initial  value 516.331455 
## iter  10 value 224.428282
## iter  20 value 185.977951
## iter  30 value 179.768415
## iter  40 value 172.984682
## iter  50 value 165.656378
## iter  60 value 158.627782
## iter  70 value 156.636274
## iter  80 value 156.624728
## final  value 156.624711 
## converged
## # weights:  106
## initial  value 473.199492 
## iter  10 value 156.286133
## iter  20 value 104.055529
## iter  30 value 80.985336
## iter  40 value 73.912804
## iter  50 value 70.605828
## iter  60 value 68.586439
## iter  70 value 62.118555
## iter  80 value 62.007224
## final  value 62.007216 
## converged
## # weights:  22
## initial  value 489.235899 
## iter  10 value 275.712993
## iter  20 value 239.740229
## iter  30 value 229.207485
## iter  40 value 220.130373
## iter  50 value 215.431832
## iter  60 value 215.217464
## final  value 215.211851 
## converged
## # weights:  64
## initial  value 499.544666 
## iter  10 value 252.348886
## iter  20 value 212.773484
## iter  30 value 184.175284
## iter  40 value 168.438998
## iter  50 value 162.619836
## iter  60 value 156.944905
## iter  70 value 156.101736
## iter  80 value 155.549441
## iter  90 value 152.378984
## iter 100 value 151.535550
## final  value 151.535550 
## stopped after 100 iterations
## # weights:  106
## initial  value 456.385258 
## iter  10 value 222.079842
## iter  20 value 177.481892
## iter  30 value 143.757907
## iter  40 value 131.550344
## iter  50 value 123.798966
## iter  60 value 118.626942
## iter  70 value 117.548392
## iter  80 value 116.960458
## iter  90 value 116.527211
## iter 100 value 115.817995
## final  value 115.817995 
## stopped after 100 iterations
## # weights:  22
## initial  value 472.285342 
## iter  10 value 298.005101
## iter  20 value 218.129858
## iter  30 value 207.918215
## iter  40 value 206.065180
## iter  50 value 205.862846
## iter  60 value 204.518224
## iter  70 value 200.883101
## iter  80 value 200.459230
## iter  90 value 200.147505
## iter 100 value 200.128299
## final  value 200.128299 
## stopped after 100 iterations
## # weights:  64
## initial  value 460.890552 
## iter  10 value 208.818823
## iter  20 value 174.824906
## iter  30 value 146.675375
## iter  40 value 129.392215
## iter  50 value 122.526398
## iter  60 value 121.015968
## iter  70 value 120.944094
## iter  80 value 120.831068
## iter  90 value 120.482434
## iter 100 value 119.875204
## final  value 119.875204 
## stopped after 100 iterations
## # weights:  106
## initial  value 492.655200 
## iter  10 value 175.377866
## iter  20 value 124.227324
## iter  30 value 90.655727
## iter  40 value 83.225776
## iter  50 value 79.288396
## iter  60 value 77.373607
## iter  70 value 75.562078
## iter  80 value 74.389147
## iter  90 value 73.531038
## iter 100 value 71.927598
## final  value 71.927598 
## stopped after 100 iterations
## # weights:  22
## initial  value 485.958618 
## iter  10 value 289.033792
## iter  20 value 210.876254
## iter  30 value 202.317137
## iter  40 value 200.387501
## iter  50 value 199.347173
## iter  60 value 194.537440
## iter  70 value 194.468697
## final  value 194.468362 
## converged
## # weights:  64
## initial  value 448.568540 
## iter  10 value 166.255337
## iter  20 value 134.017797
## iter  30 value 121.766852
## iter  40 value 113.648694
## iter  50 value 112.270249
## iter  60 value 111.574278
## iter  70 value 111.526891
## iter  80 value 111.523251
## iter  90 value 111.522683
## iter 100 value 111.522201
## final  value 111.522201 
## stopped after 100 iterations
## # weights:  106
## initial  value 441.121731 
## iter  10 value 184.985045
## iter  20 value 140.507765
## iter  30 value 105.552538
## iter  40 value 83.769909
## iter  50 value 74.757352
## iter  60 value 73.846075
## iter  70 value 73.377060
## iter  80 value 73.288694
## iter  90 value 73.274885
## iter 100 value 73.273464
## final  value 73.273464 
## stopped after 100 iterations
## # weights:  22
## initial  value 508.652887 
## iter  10 value 230.885233
## iter  20 value 211.590407
## iter  30 value 207.892669
## iter  40 value 206.989450
## final  value 206.989430 
## converged
## # weights:  64
## initial  value 493.054407 
## iter  10 value 223.310142
## iter  20 value 202.556027
## iter  30 value 192.859176
## iter  40 value 172.449231
## iter  50 value 157.365620
## iter  60 value 148.482094
## iter  70 value 146.293654
## iter  80 value 145.834359
## iter  90 value 145.826320
## final  value 145.826264 
## converged
## # weights:  106
## initial  value 636.323743 
## iter  10 value 221.578512
## iter  20 value 170.175645
## iter  30 value 158.328390
## iter  40 value 144.871619
## iter  50 value 128.312553
## iter  60 value 122.058350
## iter  70 value 119.045070
## iter  80 value 118.204872
## iter  90 value 118.015104
## iter 100 value 117.991065
## final  value 117.991065 
## stopped after 100 iterations
## # weights:  22
## initial  value 464.787177 
## iter  10 value 232.162492
## iter  20 value 206.867561
## iter  30 value 182.598126
## iter  40 value 179.808255
## iter  50 value 179.724471
## iter  60 value 179.598917
## iter  70 value 179.553491
## iter  80 value 179.528137
## iter  90 value 179.521965
## iter 100 value 179.518964
## final  value 179.518964 
## stopped after 100 iterations
## # weights:  64
## initial  value 528.452950 
## iter  10 value 211.912013
## iter  20 value 191.790034
## iter  30 value 148.452473
## iter  40 value 120.543755
## iter  50 value 95.388140
## iter  60 value 85.463612
## iter  70 value 84.240367
## iter  80 value 81.691807
## iter  90 value 80.871778
## iter 100 value 80.531458
## final  value 80.531458 
## stopped after 100 iterations
## # weights:  106
## initial  value 655.193072 
## iter  10 value 181.567649
## iter  20 value 132.436710
## iter  30 value 100.170427
## iter  40 value 83.708299
## iter  50 value 76.381914
## iter  60 value 72.011355
## iter  70 value 69.385398
## iter  80 value 67.522182
## iter  90 value 66.889163
## iter 100 value 66.336170
## final  value 66.336170 
## stopped after 100 iterations
## # weights:  22
## initial  value 473.002802 
## iter  10 value 254.550244
## iter  20 value 220.125021
## iter  30 value 213.343878
## iter  40 value 208.035052
## iter  50 value 207.752441
## iter  60 value 207.748111
## iter  70 value 207.747036
## final  value 207.747024 
## converged
## # weights:  64
## initial  value 465.647120 
## iter  10 value 196.420367
## iter  20 value 159.328410
## iter  30 value 139.218008
## iter  40 value 129.508713
## iter  50 value 126.139981
## iter  60 value 119.857951
## iter  70 value 119.670866
## final  value 119.670626 
## converged
## # weights:  106
## initial  value 474.294101 
## iter  10 value 172.332586
## iter  20 value 104.241060
## iter  30 value 74.822025
## iter  40 value 54.604939
## iter  50 value 48.475248
## iter  60 value 47.373547
## iter  70 value 47.316557
## iter  80 value 47.298114
## iter  90 value 47.293356
## iter 100 value 47.292964
## final  value 47.292964 
## stopped after 100 iterations
## # weights:  22
## initial  value 479.012649 
## iter  10 value 267.571086
## iter  20 value 231.242012
## iter  30 value 225.488835
## iter  40 value 225.374327
## final  value 225.374301 
## converged
## # weights:  64
## initial  value 550.697061 
## iter  10 value 290.093424
## iter  20 value 257.000641
## iter  30 value 211.618951
## iter  40 value 188.963707
## iter  50 value 180.297590
## iter  60 value 174.075784
## iter  70 value 171.804349
## iter  80 value 169.647503
## iter  90 value 163.556362
## iter 100 value 159.292124
## final  value 159.292124 
## stopped after 100 iterations
## # weights:  106
## initial  value 471.213601 
## iter  10 value 199.624866
## iter  20 value 154.313960
## iter  30 value 137.306791
## iter  40 value 127.751942
## iter  50 value 122.768554
## iter  60 value 120.139309
## iter  70 value 119.677513
## iter  80 value 119.548294
## iter  90 value 119.414881
## iter 100 value 119.322355
## final  value 119.322355 
## stopped after 100 iterations
## # weights:  22
## initial  value 500.586130 
## iter  10 value 255.807331
## iter  20 value 221.454065
## iter  30 value 217.933504
## iter  40 value 217.769099
## final  value 217.769009 
## converged
## # weights:  64
## initial  value 484.114592 
## iter  10 value 213.041696
## iter  20 value 180.602941
## iter  30 value 163.084309
## iter  40 value 149.300627
## iter  50 value 143.693313
## iter  60 value 141.043545
## iter  70 value 137.311115
## iter  80 value 135.201418
## iter  90 value 134.685386
## iter 100 value 134.509716
## final  value 134.509716 
## stopped after 100 iterations
## # weights:  106
## initial  value 454.631885 
## iter  10 value 169.145670
## iter  20 value 118.684094
## iter  30 value 96.685369
## iter  40 value 85.433689
## iter  50 value 79.553095
## iter  60 value 77.998135
## iter  70 value 77.005188
## iter  80 value 76.223846
## iter  90 value 75.743310
## iter 100 value 75.218996
## final  value 75.218996 
## stopped after 100 iterations
## # weights:  22
## initial  value 512.661334 
## iter  10 value 222.784278
## iter  20 value 204.436881
## iter  30 value 201.332921
## iter  40 value 200.157159
## iter  50 value 200.151879
## iter  60 value 200.149888
## iter  60 value 200.149887
## final  value 200.149887 
## converged
## # weights:  64
## initial  value 486.755415 
## iter  10 value 194.527407
## iter  20 value 150.461453
## iter  30 value 126.498714
## iter  40 value 120.923475
## iter  50 value 112.711083
## iter  60 value 109.273469
## iter  70 value 106.904499
## iter  80 value 105.037109
## iter  90 value 99.006707
## iter 100 value 98.989995
## final  value 98.989995 
## stopped after 100 iterations
## # weights:  106
## initial  value 506.081299 
## iter  10 value 195.393124
## iter  20 value 145.689935
## iter  30 value 108.998547
## iter  40 value 87.445981
## iter  50 value 76.891005
## iter  60 value 68.743211
## iter  70 value 64.962051
## iter  80 value 63.443468
## iter  90 value 60.955899
## iter 100 value 60.214066
## final  value 60.214066 
## stopped after 100 iterations
## # weights:  22
## initial  value 505.036028 
## iter  10 value 227.094140
## iter  20 value 214.161374
## iter  30 value 210.958922
## iter  40 value 210.260066
## final  value 210.259993 
## converged
## # weights:  64
## initial  value 500.205733 
## iter  10 value 202.229536
## iter  20 value 173.917903
## iter  30 value 164.074391
## iter  40 value 154.763996
## iter  50 value 150.551077
## iter  60 value 145.095692
## iter  70 value 144.021407
## iter  80 value 140.685158
## iter  90 value 140.462658
## iter 100 value 140.450562
## final  value 140.450562 
## stopped after 100 iterations
## # weights:  106
## initial  value 564.180442 
## iter  10 value 216.378474
## iter  20 value 155.460180
## iter  30 value 135.498723
## iter  40 value 123.511305
## iter  50 value 118.400481
## iter  60 value 114.260123
## iter  70 value 113.028272
## iter  80 value 112.491187
## iter  90 value 112.056016
## iter 100 value 111.875482
## final  value 111.875482 
## stopped after 100 iterations
## # weights:  22
## initial  value 482.906725 
## iter  10 value 243.179629
## iter  20 value 218.115209
## iter  30 value 194.162732
## iter  40 value 186.830670
## iter  50 value 182.542224
## iter  60 value 182.436048
## iter  70 value 182.391124
## iter  80 value 182.388481
## iter  90 value 182.384804
## iter 100 value 182.384154
## final  value 182.384154 
## stopped after 100 iterations
## # weights:  64
## initial  value 481.933321 
## iter  10 value 221.431622
## iter  20 value 178.945382
## iter  30 value 162.514686
## iter  40 value 146.220496
## iter  50 value 140.532881
## iter  60 value 136.804132
## iter  70 value 136.472407
## iter  80 value 136.285227
## iter  90 value 136.179449
## iter 100 value 135.597739
## final  value 135.597739 
## stopped after 100 iterations
## # weights:  106
## initial  value 463.124634 
## iter  10 value 191.568092
## iter  20 value 139.788341
## iter  30 value 105.511942
## iter  40 value 94.314450
## iter  50 value 92.073131
## iter  60 value 91.563090
## iter  70 value 91.207490
## iter  80 value 90.830050
## iter  90 value 90.668244
## iter 100 value 90.546249
## final  value 90.546249 
## stopped after 100 iterations
## # weights:  22
## initial  value 459.049658 
## iter  10 value 260.541760
## iter  20 value 215.038009
## iter  30 value 210.755986
## iter  40 value 210.651537
## final  value 210.644022 
## converged
## # weights:  64
## initial  value 528.826159 
## iter  10 value 273.336615
## iter  20 value 199.940823
## iter  30 value 164.822703
## iter  40 value 125.105536
## iter  50 value 96.349189
## iter  60 value 83.763860
## iter  70 value 76.952401
## iter  80 value 75.266133
## iter  90 value 70.228191
## iter 100 value 64.104371
## final  value 64.104371 
## stopped after 100 iterations
## # weights:  106
## initial  value 489.139401 
## iter  10 value 184.760169
## iter  20 value 125.091279
## iter  30 value 73.964782
## iter  40 value 53.641314
## iter  50 value 37.433603
## iter  60 value 36.241075
## iter  70 value 34.768053
## iter  80 value 33.751660
## iter  90 value 31.921795
## iter 100 value 31.734810
## final  value 31.734810 
## stopped after 100 iterations
## # weights:  22
## initial  value 497.958223 
## iter  10 value 261.380657
## iter  20 value 235.781880
## iter  30 value 223.360473
## iter  40 value 220.298545
## iter  50 value 218.279362
## iter  60 value 218.141220
## final  value 218.133342 
## converged
## # weights:  64
## initial  value 549.443253 
## iter  10 value 223.657519
## iter  20 value 179.175539
## iter  30 value 167.652391
## iter  40 value 161.090228
## iter  50 value 158.112903
## iter  60 value 157.650821
## iter  70 value 157.276267
## iter  80 value 157.160176
## iter  90 value 157.158952
## iter  90 value 157.158951
## iter  90 value 157.158951
## final  value 157.158951 
## converged
## # weights:  106
## initial  value 524.854949 
## iter  10 value 205.764840
## iter  20 value 155.445898
## iter  30 value 131.120039
## iter  40 value 118.503570
## iter  50 value 114.558129
## iter  60 value 113.007561
## iter  70 value 112.407700
## iter  80 value 112.145748
## iter  90 value 112.028190
## iter 100 value 111.967715
## final  value 111.967715 
## stopped after 100 iterations
## # weights:  22
## initial  value 510.642206 
## iter  10 value 219.904482
## iter  20 value 212.212300
## iter  30 value 211.404525
## iter  40 value 211.116148
## iter  50 value 210.731995
## iter  60 value 210.655711
## final  value 210.655629 
## converged
## # weights:  64
## initial  value 474.765713 
## iter  10 value 225.902176
## iter  20 value 163.820973
## iter  30 value 130.281733
## iter  40 value 120.823433
## iter  50 value 119.783286
## iter  60 value 119.287789
## iter  70 value 119.079104
## iter  80 value 119.035099
## iter  90 value 119.007853
## iter 100 value 118.981954
## final  value 118.981954 
## stopped after 100 iterations
## # weights:  106
## initial  value 604.413037 
## iter  10 value 183.819594
## iter  20 value 112.652500
## iter  30 value 93.367141
## iter  40 value 79.625730
## iter  50 value 73.703611
## iter  60 value 69.832503
## iter  70 value 65.583595
## iter  80 value 63.599145
## iter  90 value 62.854656
## iter 100 value 62.538296
## final  value 62.538296 
## stopped after 100 iterations
## # weights:  22
## initial  value 498.862017 
## iter  10 value 299.466782
## iter  20 value 240.266139
## iter  30 value 212.568255
## iter  40 value 211.565390
## iter  50 value 211.445103
## iter  60 value 211.440113
## iter  70 value 211.432177
## final  value 211.431729 
## converged
## # weights:  64
## initial  value 557.733086 
## iter  10 value 242.037974
## iter  20 value 173.233903
## iter  30 value 151.087708
## iter  40 value 134.283771
## iter  50 value 118.846379
## iter  60 value 112.004672
## iter  70 value 107.905022
## iter  80 value 105.652060
## iter  90 value 103.732069
## iter 100 value 102.750599
## final  value 102.750599 
## stopped after 100 iterations
## # weights:  106
## initial  value 471.369271 
## iter  10 value 178.760950
## iter  20 value 124.860426
## iter  30 value 96.253996
## iter  40 value 86.230675
## iter  50 value 82.610699
## iter  60 value 80.564343
## iter  70 value 80.271508
## iter  80 value 80.250363
## iter  90 value 80.248700
## final  value 80.248561 
## converged
## # weights:  22
## initial  value 470.901098 
## iter  10 value 273.666198
## iter  20 value 226.959341
## iter  30 value 219.641302
## iter  40 value 219.362926
## iter  40 value 219.362925
## iter  40 value 219.362925
## final  value 219.362925 
## converged
## # weights:  64
## initial  value 581.741410 
## iter  10 value 207.507834
## iter  20 value 172.232015
## iter  30 value 154.140309
## iter  40 value 144.096926
## iter  50 value 141.664061
## iter  60 value 141.084705
## iter  70 value 140.928959
## iter  80 value 140.854244
## iter  90 value 140.847198
## iter 100 value 140.846741
## final  value 140.846741 
## stopped after 100 iterations
## # weights:  106
## initial  value 509.920658 
## iter  10 value 260.556325
## iter  20 value 186.206770
## iter  30 value 155.155795
## iter  40 value 134.123620
## iter  50 value 124.667312
## iter  60 value 120.565237
## iter  70 value 117.915082
## iter  80 value 116.827557
## iter  90 value 116.320336
## iter 100 value 116.099216
## final  value 116.099216 
## stopped after 100 iterations
## # weights:  22
## initial  value 526.537460 
## iter  10 value 239.384390
## iter  20 value 200.348189
## iter  30 value 192.493574
## iter  40 value 186.323234
## iter  50 value 185.953616
## iter  60 value 185.833334
## iter  70 value 185.820309
## iter  80 value 185.817634
## iter  90 value 185.808300
## final  value 185.805323 
## converged
## # weights:  64
## initial  value 510.727682 
## iter  10 value 212.770625
## iter  20 value 155.767218
## iter  30 value 135.948027
## iter  40 value 123.247808
## iter  50 value 114.937054
## iter  60 value 114.483025
## iter  70 value 114.360049
## iter  80 value 114.091058
## iter  90 value 113.588893
## iter 100 value 113.028473
## final  value 113.028473 
## stopped after 100 iterations
## # weights:  106
## initial  value 464.170595 
## iter  10 value 162.667999
## iter  20 value 121.100151
## iter  30 value 96.286568
## iter  40 value 86.799725
## iter  50 value 85.564395
## iter  60 value 83.822232
## iter  70 value 82.669519
## iter  80 value 82.169848
## iter  90 value 81.408269
## iter 100 value 81.299445
## final  value 81.299445 
## stopped after 100 iterations
## # weights:  22
## initial  value 478.943322 
## iter  10 value 303.644592
## iter  20 value 233.434389
## iter  30 value 216.651102
## iter  40 value 200.838559
## iter  50 value 193.716835
## iter  60 value 193.127435
## iter  70 value 191.035474
## iter  80 value 190.025443
## iter  90 value 190.013505
## final  value 190.013499 
## converged
## # weights:  64
## initial  value 504.425690 
## iter  10 value 203.363398
## iter  20 value 167.238755
## iter  30 value 156.679250
## iter  40 value 148.414878
## iter  50 value 143.291764
## iter  60 value 142.502934
## iter  70 value 138.499553
## iter  80 value 138.037698
## iter  90 value 137.918248
## iter 100 value 137.904168
## final  value 137.904168 
## stopped after 100 iterations
## # weights:  106
## initial  value 481.477245 
## iter  10 value 192.576861
## iter  20 value 132.079658
## iter  30 value 110.281384
## iter  40 value 103.372464
## iter  50 value 99.385697
## iter  60 value 97.419461
## iter  70 value 93.168799
## iter  80 value 90.623653
## iter  90 value 89.933215
## iter 100 value 89.622678
## final  value 89.622678 
## stopped after 100 iterations
## # weights:  22
## initial  value 491.610763 
## iter  10 value 297.501858
## iter  20 value 231.466900
## iter  30 value 222.744408
## iter  40 value 220.296143
## iter  50 value 219.303035
## iter  60 value 219.299316
## final  value 219.299157 
## converged
## # weights:  64
## initial  value 590.487792 
## iter  10 value 230.369032
## iter  20 value 185.169380
## iter  30 value 154.852215
## iter  40 value 143.564923
## iter  50 value 140.797697
## iter  60 value 139.973274
## iter  70 value 139.875502
## iter  80 value 139.858280
## final  value 139.858065 
## converged
## # weights:  106
## initial  value 493.235468 
## iter  10 value 214.190470
## iter  20 value 174.355715
## iter  30 value 149.328178
## iter  40 value 136.980767
## iter  50 value 127.421984
## iter  60 value 118.339088
## iter  70 value 113.804229
## iter  80 value 110.117592
## iter  90 value 109.269797
## iter 100 value 108.541212
## final  value 108.541212 
## stopped after 100 iterations
## # weights:  22
## initial  value 508.058447 
## iter  10 value 226.035147
## iter  20 value 216.722474
## iter  30 value 209.809771
## iter  40 value 199.343131
## iter  50 value 199.076756
## iter  60 value 198.895451
## iter  70 value 198.774799
## iter  80 value 198.743152
## iter  90 value 198.727584
## iter 100 value 198.719190
## final  value 198.719190 
## stopped after 100 iterations
## # weights:  64
## initial  value 486.084129 
## iter  10 value 264.594565
## iter  20 value 174.922229
## iter  30 value 143.652584
## iter  40 value 132.070591
## iter  50 value 125.437346
## iter  60 value 123.966746
## iter  70 value 121.983281
## iter  80 value 121.139860
## iter  90 value 117.857999
## iter 100 value 117.044948
## final  value 117.044948 
## stopped after 100 iterations
## # weights:  106
## initial  value 487.202711 
## iter  10 value 217.201177
## iter  20 value 146.203367
## iter  30 value 103.108146
## iter  40 value 75.482401
## iter  50 value 68.185682
## iter  60 value 65.076375
## iter  70 value 63.431612
## iter  80 value 62.788383
## iter  90 value 62.404192
## iter 100 value 61.769077
## final  value 61.769077 
## stopped after 100 iterations
## # weights:  22
## initial  value 475.013364 
## iter  10 value 238.069361
## iter  20 value 215.840330
## iter  30 value 204.796115
## iter  40 value 202.867380
## iter  50 value 200.810843
## iter  60 value 200.792372
## final  value 200.792325 
## converged
## # weights:  64
## initial  value 503.526815 
## iter  10 value 225.030535
## iter  20 value 184.454333
## iter  30 value 153.955323
## iter  40 value 132.045508
## iter  50 value 128.334680
## iter  60 value 126.052215
## iter  70 value 122.807360
## iter  80 value 121.102918
## iter  90 value 118.946167
## iter 100 value 116.190889
## final  value 116.190889 
## stopped after 100 iterations
## # weights:  106
## initial  value 481.979571 
## iter  10 value 191.182122
## iter  20 value 144.133026
## iter  30 value 111.646488
## iter  40 value 99.792482
## iter  50 value 96.750432
## iter  60 value 96.676217
## iter  70 value 96.318028
## final  value 96.317944 
## converged
## # weights:  22
## initial  value 493.541199 
## iter  10 value 263.215267
## iter  20 value 229.177658
## iter  30 value 218.805523
## iter  40 value 213.594430
## iter  50 value 213.193953
## iter  60 value 213.173921
## final  value 213.173455 
## converged
## # weights:  64
## initial  value 481.741090 
## iter  10 value 232.850079
## iter  20 value 202.095248
## iter  30 value 193.777780
## iter  40 value 191.750873
## iter  50 value 191.298838
## iter  60 value 191.080912
## iter  70 value 191.040031
## final  value 191.039926 
## converged
## # weights:  106
## initial  value 524.407039 
## iter  10 value 198.069559
## iter  20 value 150.235219
## iter  30 value 132.806235
## iter  40 value 125.892585
## iter  50 value 120.396848
## iter  60 value 119.874175
## iter  70 value 119.656322
## iter  80 value 119.593056
## iter  90 value 119.583812
## final  value 119.583679 
## converged
## # weights:  22
## initial  value 507.877374 
## iter  10 value 290.469724
## iter  20 value 214.263341
## iter  30 value 207.032191
## iter  40 value 206.839432
## final  value 206.839427 
## converged
## # weights:  64
## initial  value 544.332483 
## iter  10 value 263.716679
## iter  20 value 169.070212
## iter  30 value 158.245439
## iter  40 value 149.112045
## iter  50 value 147.084084
## iter  60 value 145.484352
## iter  70 value 144.433559
## iter  80 value 143.274906
## iter  90 value 141.402830
## iter 100 value 136.991968
## final  value 136.991968 
## stopped after 100 iterations
## # weights:  106
## initial  value 563.791378 
## iter  10 value 186.794851
## iter  20 value 125.431435
## iter  30 value 99.728153
## iter  40 value 92.671367
## iter  50 value 90.174255
## iter  60 value 88.589260
## iter  70 value 88.293344
## iter  80 value 88.069232
## iter  90 value 87.739664
## iter 100 value 87.470174
## final  value 87.470174 
## stopped after 100 iterations
## # weights:  22
## initial  value 493.392245 
## iter  10 value 266.304796
## iter  20 value 223.406217
## iter  30 value 216.174215
## iter  40 value 215.590850
## final  value 215.590027 
## converged
## # weights:  64
## initial  value 505.904683 
## iter  10 value 194.843027
## iter  20 value 169.830163
## iter  30 value 138.160023
## iter  40 value 127.709103
## iter  50 value 123.810311
## iter  60 value 122.881469
## iter  70 value 122.842299
## iter  80 value 122.828871
## iter  90 value 122.828023
## iter  90 value 122.828022
## iter  90 value 122.828022
## final  value 122.828022 
## converged
## # weights:  106
## initial  value 519.962478 
## iter  10 value 159.376976
## iter  20 value 111.450236
## iter  30 value 79.754971
## iter  40 value 74.169728
## iter  50 value 71.136359
## iter  60 value 69.426355
## iter  70 value 67.942746
## iter  80 value 67.061105
## iter  90 value 60.141389
## iter 100 value 57.756187
## final  value 57.756187 
## stopped after 100 iterations
## # weights:  22
## initial  value 508.863348 
## iter  10 value 370.835073
## iter  20 value 233.644507
## iter  30 value 204.842228
## iter  40 value 204.556430
## iter  50 value 204.244145
## iter  60 value 204.182208
## iter  70 value 204.180034
## iter  70 value 204.180034
## iter  70 value 204.180034
## final  value 204.180034 
## converged
## # weights:  64
## initial  value 470.396392 
## iter  10 value 213.935010
## iter  20 value 181.813046
## iter  30 value 167.667879
## iter  40 value 159.768107
## iter  50 value 157.895349
## iter  60 value 157.200178
## iter  70 value 156.881004
## iter  80 value 156.341712
## iter  90 value 156.203455
## iter 100 value 156.193871
## final  value 156.193871 
## stopped after 100 iterations
## # weights:  106
## initial  value 533.732556 
## iter  10 value 202.080702
## iter  20 value 161.682627
## iter  30 value 135.965920
## iter  40 value 120.988542
## iter  50 value 115.270572
## iter  60 value 113.093571
## iter  70 value 112.908044
## iter  80 value 112.881448
## iter  90 value 112.726872
## iter 100 value 112.548979
## final  value 112.548979 
## stopped after 100 iterations
## # weights:  22
## initial  value 488.543920 
## iter  10 value 226.018639
## iter  20 value 200.211022
## iter  30 value 197.781506
## iter  40 value 196.858740
## iter  50 value 195.607146
## iter  60 value 194.959335
## iter  70 value 184.837244
## iter  80 value 184.335744
## iter  90 value 184.155004
## iter 100 value 184.027157
## final  value 184.027157 
## stopped after 100 iterations
## # weights:  64
## initial  value 516.930127 
## iter  10 value 292.804431
## iter  20 value 171.735604
## iter  30 value 141.059408
## iter  40 value 130.526506
## iter  50 value 126.606771
## iter  60 value 124.958605
## iter  70 value 123.032054
## iter  80 value 122.535406
## iter  90 value 121.913148
## iter 100 value 120.353358
## final  value 120.353358 
## stopped after 100 iterations
## # weights:  106
## initial  value 477.420078 
## iter  10 value 187.241350
## iter  20 value 99.699646
## iter  30 value 68.576413
## iter  40 value 57.321711
## iter  50 value 51.631497
## iter  60 value 50.728893
## iter  70 value 49.874032
## iter  80 value 48.843523
## iter  90 value 48.587882
## iter 100 value 48.363431
## final  value 48.363431 
## stopped after 100 iterations
## # weights:  22
## initial  value 512.264548 
## iter  10 value 308.319385
## iter  20 value 214.379586
## iter  30 value 196.265086
## iter  40 value 194.960401
## iter  50 value 194.392275
## iter  60 value 194.378774
## iter  70 value 194.129417
## iter  80 value 193.744297
## iter  90 value 193.611606
## final  value 193.596814 
## converged
## # weights:  64
## initial  value 540.207663 
## iter  10 value 210.177801
## iter  20 value 147.647680
## iter  30 value 122.199455
## iter  40 value 99.280726
## iter  50 value 94.036752
## iter  60 value 93.332485
## iter  70 value 93.264607
## final  value 93.264058 
## converged
## # weights:  106
## initial  value 485.975036 
## iter  10 value 204.601498
## iter  20 value 109.195204
## iter  30 value 93.079074
## iter  40 value 78.300774
## iter  50 value 70.843371
## iter  60 value 69.016262
## iter  70 value 67.798460
## final  value 67.133159 
## converged
## # weights:  22
## initial  value 485.237643 
## iter  10 value 276.640700
## iter  20 value 230.346518
## iter  30 value 215.742995
## iter  40 value 210.656686
## iter  50 value 205.561172
## iter  60 value 204.689651
## iter  70 value 204.628005
## iter  70 value 204.628004
## iter  70 value 204.628004
## final  value 204.628004 
## converged
## # weights:  64
## initial  value 531.799127 
## iter  10 value 213.091481
## iter  20 value 184.567115
## iter  30 value 178.884787
## iter  40 value 176.613577
## iter  50 value 175.912640
## iter  60 value 175.312365
## iter  70 value 175.200508
## iter  80 value 175.198543
## final  value 175.198534 
## converged
## # weights:  106
## initial  value 482.200487 
## iter  10 value 195.338531
## iter  20 value 149.629876
## iter  30 value 135.282359
## iter  40 value 126.054066
## iter  50 value 119.328596
## iter  60 value 113.227013
## iter  70 value 111.603861
## iter  80 value 111.099361
## iter  90 value 110.875258
## iter 100 value 110.725752
## final  value 110.725752 
## stopped after 100 iterations
## # weights:  22
## initial  value 504.225063 
## iter  10 value 220.412856
## iter  20 value 201.161873
## iter  30 value 200.262202
## iter  40 value 198.692326
## iter  50 value 196.786076
## iter  60 value 196.689774
## iter  70 value 196.650378
## iter  80 value 196.633148
## iter  90 value 196.631213
## iter 100 value 196.630505
## final  value 196.630505 
## stopped after 100 iterations
## # weights:  64
## initial  value 463.371822 
## iter  10 value 195.406991
## iter  20 value 158.838900
## iter  30 value 131.819260
## iter  40 value 117.413773
## iter  50 value 114.019130
## iter  60 value 113.750587
## iter  70 value 113.640732
## iter  80 value 113.559342
## iter  90 value 113.157893
## iter 100 value 112.761544
## final  value 112.761544 
## stopped after 100 iterations
## # weights:  106
## initial  value 498.841895 
## iter  10 value 191.534151
## iter  20 value 138.062561
## iter  30 value 105.898363
## iter  40 value 82.478173
## iter  50 value 73.700887
## iter  60 value 72.026076
## iter  70 value 71.375569
## iter  80 value 71.097939
## iter  90 value 70.962397
## iter 100 value 70.726564
## final  value 70.726564 
## stopped after 100 iterations
## # weights:  22
## initial  value 499.703556 
## iter  10 value 268.161353
## iter  20 value 212.329567
## iter  30 value 206.167253
## iter  40 value 204.666061
## iter  50 value 204.212236
## iter  60 value 203.969957
## iter  70 value 203.503030
## iter  80 value 203.502262
## final  value 203.502244 
## converged
## # weights:  64
## initial  value 490.415963 
## iter  10 value 233.060543
## iter  20 value 174.843038
## iter  30 value 152.375654
## iter  40 value 136.830135
## iter  50 value 121.359672
## iter  60 value 113.049020
## iter  70 value 112.131029
## iter  80 value 112.100080
## iter  90 value 112.097458
## iter 100 value 112.089236
## final  value 112.089236 
## stopped after 100 iterations
## # weights:  106
## initial  value 466.315920 
## iter  10 value 175.597917
## iter  20 value 108.166057
## iter  30 value 85.651209
## iter  40 value 76.305458
## iter  50 value 65.452576
## iter  60 value 63.110811
## iter  70 value 62.739244
## iter  80 value 62.390764
## iter  90 value 62.363669
## iter 100 value 62.329649
## final  value 62.329649 
## stopped after 100 iterations
## # weights:  22
## initial  value 495.733276 
## iter  10 value 280.742215
## iter  20 value 217.278616
## iter  30 value 211.415291
## iter  40 value 210.605703
## final  value 210.541856 
## converged
## # weights:  64
## initial  value 559.536771 
## iter  10 value 212.635279
## iter  20 value 178.431693
## iter  30 value 166.366729
## iter  40 value 159.609868
## iter  50 value 153.433546
## iter  60 value 148.105698
## iter  70 value 145.746691
## iter  80 value 144.987663
## iter  90 value 144.901028
## iter 100 value 144.895965
## final  value 144.895965 
## stopped after 100 iterations
## # weights:  106
## initial  value 548.117159 
## iter  10 value 275.615304
## iter  20 value 208.911601
## iter  30 value 174.397581
## iter  40 value 152.972811
## iter  50 value 139.839463
## iter  60 value 130.241130
## iter  70 value 124.975059
## iter  80 value 121.494684
## iter  90 value 120.311465
## iter 100 value 118.630161
## final  value 118.630161 
## stopped after 100 iterations
## # weights:  22
## initial  value 525.591056 
## iter  10 value 217.930522
## iter  20 value 210.575749
## iter  30 value 208.889351
## iter  40 value 205.875679
## iter  50 value 201.350803
## iter  60 value 200.345761
## iter  70 value 198.836643
## iter  80 value 198.541307
## iter  90 value 197.636238
## iter 100 value 197.564516
## final  value 197.564516 
## stopped after 100 iterations
## # weights:  64
## initial  value 516.533878 
## iter  10 value 210.786825
## iter  20 value 153.493145
## iter  30 value 141.890217
## iter  40 value 136.129299
## iter  50 value 135.165209
## iter  60 value 134.803325
## iter  70 value 134.387002
## iter  80 value 134.078877
## iter  90 value 133.875286
## iter 100 value 133.805824
## final  value 133.805824 
## stopped after 100 iterations
## # weights:  106
## initial  value 505.534600 
## iter  10 value 195.254341
## iter  20 value 119.938637
## iter  30 value 88.797527
## iter  40 value 70.812666
## iter  50 value 63.966822
## iter  60 value 63.336507
## iter  70 value 62.683763
## iter  80 value 62.039017
## iter  90 value 60.487278
## iter 100 value 59.947112
## final  value 59.947112 
## stopped after 100 iterations
## # weights:  22
## initial  value 510.032681 
## iter  10 value 319.295701
## iter  20 value 220.228014
## iter  30 value 209.598066
## iter  40 value 208.418128
## iter  50 value 204.642020
## iter  60 value 203.957278
## iter  70 value 203.457815
## iter  80 value 202.401200
## iter  90 value 201.021123
## iter 100 value 200.500799
## final  value 200.500799 
## stopped after 100 iterations
## # weights:  64
## initial  value 552.716803 
## iter  10 value 201.960864
## iter  20 value 178.852930
## iter  30 value 170.543727
## iter  40 value 163.718565
## iter  50 value 160.747801
## iter  60 value 158.543823
## iter  70 value 156.376211
## iter  80 value 155.775612
## iter  90 value 155.281461
## iter 100 value 153.718078
## final  value 153.718078 
## stopped after 100 iterations
## # weights:  106
## initial  value 500.721573 
## iter  10 value 197.642995
## iter  20 value 129.413022
## iter  30 value 92.064396
## iter  40 value 70.161301
## iter  50 value 64.900401
## iter  60 value 63.053354
## iter  70 value 61.139999
## iter  80 value 60.854812
## iter  90 value 60.827831
## iter 100 value 60.826926
## final  value 60.826926 
## stopped after 100 iterations
## # weights:  22
## initial  value 466.586096 
## iter  10 value 243.779287
## iter  20 value 218.838017
## iter  30 value 215.966281
## final  value 215.965957 
## converged
## # weights:  64
## initial  value 470.444755 
## iter  10 value 203.038659
## iter  20 value 188.601364
## iter  30 value 185.693574
## iter  40 value 184.495414
## iter  50 value 182.789926
## iter  60 value 182.537916
## iter  70 value 182.020744
## iter  80 value 182.006688
## final  value 182.006578 
## converged
## # weights:  106
## initial  value 495.592340 
## iter  10 value 202.537850
## iter  20 value 159.684958
## iter  30 value 145.631400
## iter  40 value 129.150611
## iter  50 value 121.806181
## iter  60 value 120.059218
## iter  70 value 118.890030
## iter  80 value 117.542075
## iter  90 value 117.006764
## iter 100 value 116.585523
## final  value 116.585523 
## stopped after 100 iterations
## # weights:  22
## initial  value 501.188666 
## iter  10 value 226.781925
## iter  20 value 213.502880
## iter  30 value 201.318319
## iter  40 value 197.369655
## iter  50 value 197.121607
## iter  60 value 196.925382
## iter  70 value 196.869487
## iter  80 value 196.860268
## iter  90 value 196.848264
## iter 100 value 196.841948
## final  value 196.841948 
## stopped after 100 iterations
## # weights:  64
## initial  value 499.984077 
## iter  10 value 218.317071
## iter  20 value 181.147304
## iter  30 value 164.005233
## iter  40 value 159.103410
## iter  50 value 156.897028
## iter  60 value 155.101053
## iter  70 value 153.982613
## iter  80 value 153.798012
## iter  90 value 153.727221
## iter 100 value 153.544288
## final  value 153.544288 
## stopped after 100 iterations
## # weights:  106
## initial  value 444.902750 
## iter  10 value 183.803618
## iter  20 value 127.335277
## iter  30 value 105.551072
## iter  40 value 98.381020
## iter  50 value 96.136906
## iter  60 value 95.633890
## iter  70 value 95.265146
## iter  80 value 95.034904
## iter  90 value 94.841400
## iter 100 value 94.546322
## final  value 94.546322 
## stopped after 100 iterations
## # weights:  22
## initial  value 460.769218 
## iter  10 value 225.517598
## iter  20 value 206.018643
## iter  30 value 204.141912
## iter  40 value 204.038007
## final  value 204.037852 
## converged
## # weights:  64
## initial  value 558.323940 
## iter  10 value 200.039958
## iter  20 value 153.784619
## iter  30 value 123.504774
## iter  40 value 112.541727
## iter  50 value 107.044187
## iter  60 value 103.131450
## iter  70 value 100.922105
## iter  80 value 97.692639
## iter  90 value 97.611863
## iter 100 value 97.585743
## final  value 97.585743 
## stopped after 100 iterations
## # weights:  106
## initial  value 542.516726 
## iter  10 value 193.496635
## iter  20 value 130.841415
## iter  30 value 116.753482
## iter  40 value 106.460962
## iter  50 value 98.258363
## iter  60 value 93.432201
## iter  70 value 92.520559
## iter  80 value 88.920585
## iter  90 value 67.757902
## iter 100 value 62.750318
## final  value 62.750318 
## stopped after 100 iterations
## # weights:  22
## initial  value 521.469238 
## iter  10 value 228.309291
## iter  20 value 219.131928
## iter  30 value 218.618411
## final  value 218.618140 
## converged
## # weights:  64
## initial  value 466.347150 
## iter  10 value 203.468004
## iter  20 value 182.777270
## iter  30 value 167.476410
## iter  40 value 159.678403
## iter  50 value 156.845381
## iter  60 value 155.563022
## iter  70 value 154.504802
## iter  80 value 154.260185
## iter  90 value 154.258572
## final  value 154.258570 
## converged
## # weights:  106
## initial  value 530.655188 
## iter  10 value 222.374467
## iter  20 value 171.935321
## iter  30 value 145.715803
## iter  40 value 131.438522
## iter  50 value 120.452186
## iter  60 value 114.334985
## iter  70 value 112.249441
## iter  80 value 111.480209
## iter  90 value 111.352968
## iter 100 value 111.293285
## final  value 111.293285 
## stopped after 100 iterations
## # weights:  22
## initial  value 466.041507 
## iter  10 value 243.693451
## iter  20 value 221.753153
## iter  30 value 205.026749
## iter  40 value 194.378145
## iter  50 value 187.480598
## iter  60 value 186.882225
## iter  70 value 186.752125
## iter  80 value 186.713369
## iter  90 value 186.698751
## iter 100 value 186.688206
## final  value 186.688206 
## stopped after 100 iterations
## # weights:  64
## initial  value 475.943212 
## iter  10 value 187.775965
## iter  20 value 139.328852
## iter  30 value 126.060813
## iter  40 value 120.887415
## iter  50 value 118.274856
## iter  60 value 116.940043
## iter  70 value 116.341458
## iter  80 value 115.760633
## iter  90 value 115.501741
## iter 100 value 115.314295
## final  value 115.314295 
## stopped after 100 iterations
## # weights:  106
## initial  value 439.602392 
## iter  10 value 183.684718
## iter  20 value 128.091829
## iter  30 value 103.144530
## iter  40 value 90.615458
## iter  50 value 83.999400
## iter  60 value 76.764294
## iter  70 value 72.241377
## iter  80 value 70.761675
## iter  90 value 70.310353
## iter 100 value 69.428922
## final  value 69.428922 
## stopped after 100 iterations
## # weights:  22
## initial  value 495.887871 
## iter  10 value 233.470678
## iter  20 value 204.827249
## iter  30 value 203.537278
## iter  40 value 198.936542
## iter  50 value 196.446884
## iter  60 value 196.388446
## iter  70 value 196.384367
## iter  80 value 196.379858
## iter  90 value 196.378516
## iter 100 value 196.377827
## final  value 196.377827 
## stopped after 100 iterations
## # weights:  64
## initial  value 474.549515 
## iter  10 value 199.242242
## iter  20 value 167.388311
## iter  30 value 133.026677
## iter  40 value 110.241977
## iter  50 value 102.099602
## iter  60 value 101.083295
## iter  70 value 101.062102
## final  value 101.061966 
## converged
## # weights:  106
## initial  value 494.157228 
## iter  10 value 172.038966
## iter  20 value 111.074683
## iter  30 value 85.670895
## iter  40 value 77.442170
## iter  50 value 73.548999
## iter  60 value 72.345411
## iter  70 value 70.877525
## iter  80 value 70.094726
## iter  90 value 69.527954
## iter 100 value 69.136815
## final  value 69.136815 
## stopped after 100 iterations
## # weights:  22
## initial  value 490.032230 
## iter  10 value 273.397787
## iter  20 value 220.637992
## iter  30 value 214.896801
## iter  40 value 212.830515
## iter  50 value 212.747886
## iter  60 value 212.747476
## final  value 212.747454 
## converged
## # weights:  64
## initial  value 616.359369 
## iter  10 value 245.180830
## iter  20 value 198.061318
## iter  30 value 174.838843
## iter  40 value 167.031488
## iter  50 value 163.596302
## iter  60 value 161.684171
## iter  70 value 161.465366
## iter  80 value 161.464145
## final  value 161.464138 
## converged
## # weights:  106
## initial  value 500.505675 
## iter  10 value 190.705127
## iter  20 value 158.452732
## iter  30 value 145.427491
## iter  40 value 132.727535
## iter  50 value 128.182828
## iter  60 value 126.895693
## iter  70 value 126.482319
## iter  80 value 125.889491
## iter  90 value 124.855151
## iter 100 value 114.873608
## final  value 114.873608 
## stopped after 100 iterations
## # weights:  22
## initial  value 539.658303 
## iter  10 value 365.971003
## iter  20 value 232.462207
## iter  30 value 204.113606
## iter  40 value 200.707014
## iter  50 value 199.552989
## iter  60 value 199.498281
## iter  70 value 199.457016
## iter  80 value 199.452874
## iter  90 value 199.452630
## final  value 199.452347 
## converged
## # weights:  64
## initial  value 553.371213 
## iter  10 value 192.109616
## iter  20 value 128.883149
## iter  30 value 104.108135
## iter  40 value 96.644060
## iter  50 value 94.207858
## iter  60 value 93.582641
## iter  70 value 93.421909
## iter  80 value 93.227196
## iter  90 value 93.055376
## iter 100 value 92.966371
## final  value 92.966371 
## stopped after 100 iterations
## # weights:  106
## initial  value 558.061680 
## iter  10 value 207.052268
## iter  20 value 141.998723
## iter  30 value 101.745090
## iter  40 value 84.383474
## iter  50 value 65.087440
## iter  60 value 59.631691
## iter  70 value 59.164501
## iter  80 value 58.607318
## iter  90 value 58.208268
## iter 100 value 57.603986
## final  value 57.603986 
## stopped after 100 iterations
## # weights:  22
## initial  value 517.260687 
## iter  10 value 214.832727
## iter  20 value 200.832568
## iter  30 value 197.504809
## iter  40 value 194.388286
## iter  50 value 193.001591
## iter  60 value 192.127394
## iter  70 value 191.767595
## iter  80 value 191.490128
## iter  90 value 191.197831
## iter 100 value 191.100024
## final  value 191.100024 
## stopped after 100 iterations
## # weights:  64
## initial  value 504.432573 
## iter  10 value 192.202797
## iter  20 value 154.529445
## iter  30 value 136.306796
## iter  40 value 123.002968
## iter  50 value 120.132422
## iter  60 value 116.987311
## iter  70 value 108.033770
## iter  80 value 106.299013
## iter  90 value 105.812124
## iter 100 value 101.523016
## final  value 101.523016 
## stopped after 100 iterations
## # weights:  106
## initial  value 470.549507 
## iter  10 value 166.069447
## iter  20 value 98.117425
## iter  30 value 83.445177
## iter  40 value 73.562346
## iter  50 value 60.609468
## iter  60 value 57.269380
## iter  70 value 55.719243
## iter  80 value 54.764391
## iter  90 value 54.365289
## iter 100 value 54.166351
## final  value 54.166351 
## stopped after 100 iterations
## # weights:  22
## initial  value 483.855021 
## iter  10 value 248.192854
## iter  20 value 216.219770
## iter  30 value 214.080375
## iter  40 value 211.383292
## iter  50 value 209.295813
## iter  60 value 209.203290
## iter  70 value 209.199916
## iter  80 value 208.224093
## iter  90 value 207.516575
## final  value 207.486880 
## converged
## # weights:  64
## initial  value 493.977726 
## iter  10 value 240.326357
## iter  20 value 209.138377
## iter  30 value 185.129986
## iter  40 value 177.733349
## iter  50 value 175.298619
## iter  60 value 173.537918
## iter  70 value 172.263970
## iter  80 value 172.208037
## iter  90 value 172.205059
## final  value 172.205037 
## converged
## # weights:  106
## initial  value 469.914008 
## iter  10 value 180.087947
## iter  20 value 156.005138
## iter  30 value 144.891519
## iter  40 value 137.148415
## iter  50 value 132.001481
## iter  60 value 130.257066
## iter  70 value 129.328376
## iter  80 value 128.993639
## iter  90 value 128.894498
## iter 100 value 128.893081
## final  value 128.893081 
## stopped after 100 iterations
## # weights:  22
## initial  value 511.562962 
## iter  10 value 273.997377
## iter  20 value 211.475626
## iter  30 value 202.626295
## iter  40 value 202.604618
## final  value 202.597819 
## converged
## # weights:  64
## initial  value 480.983721 
## iter  10 value 183.839869
## iter  20 value 149.609263
## iter  30 value 140.337857
## iter  40 value 130.996405
## iter  50 value 126.605207
## iter  60 value 125.855225
## iter  70 value 125.700967
## iter  80 value 125.456178
## iter  90 value 125.050985
## iter 100 value 124.834724
## final  value 124.834724 
## stopped after 100 iterations
## # weights:  106
## initial  value 524.359174 
## iter  10 value 201.542434
## iter  20 value 155.830591
## iter  30 value 128.731433
## iter  40 value 115.868328
## iter  50 value 111.474402
## iter  60 value 110.013306
## iter  70 value 109.372006
## iter  80 value 108.268515
## iter  90 value 107.946647
## iter 100 value 107.278119
## final  value 107.278119 
## stopped after 100 iterations
## # weights:  22
## initial  value 515.540987 
## iter  10 value 231.276887
## iter  20 value 220.046829
## iter  30 value 211.449300
## iter  40 value 202.205865
## iter  50 value 195.427370
## iter  60 value 193.751781
## iter  70 value 193.745395
## final  value 193.745389 
## converged
## # weights:  64
## initial  value 488.614127 
## iter  10 value 188.637956
## iter  20 value 126.391114
## iter  30 value 103.564100
## iter  40 value 98.815252
## iter  50 value 96.935238
## iter  60 value 96.064148
## iter  70 value 95.914349
## iter  80 value 95.879844
## iter  90 value 95.877846
## final  value 95.877843 
## converged
## # weights:  106
## initial  value 455.362107 
## iter  10 value 173.628678
## iter  20 value 127.943760
## iter  30 value 91.652855
## iter  40 value 78.009491
## iter  50 value 71.390142
## iter  60 value 65.641581
## iter  70 value 64.921543
## iter  80 value 64.897362
## iter  90 value 64.895355
## iter 100 value 64.893844
## final  value 64.893844 
## stopped after 100 iterations
## # weights:  22
## initial  value 473.045754 
## iter  10 value 244.791190
## iter  20 value 227.666512
## iter  30 value 213.895064
## iter  40 value 208.576056
## iter  50 value 208.315612
## iter  60 value 208.313728
## final  value 208.313654 
## converged
## # weights:  64
## initial  value 451.075526 
## iter  10 value 241.432527
## iter  20 value 188.822706
## iter  30 value 165.534025
## iter  40 value 161.727365
## iter  50 value 160.494691
## iter  60 value 144.596981
## iter  70 value 139.388061
## iter  80 value 138.058361
## iter  90 value 136.150015
## iter 100 value 135.268302
## final  value 135.268302 
## stopped after 100 iterations
## # weights:  106
## initial  value 471.562941 
## iter  10 value 181.358411
## iter  20 value 156.331927
## iter  30 value 145.312772
## iter  40 value 130.165462
## iter  50 value 123.530406
## iter  60 value 120.941544
## iter  70 value 120.210719
## iter  80 value 119.879293
## iter  90 value 119.830426
## iter 100 value 119.823286
## final  value 119.823286 
## stopped after 100 iterations
## # weights:  22
## initial  value 483.848293 
## iter  10 value 301.682206
## iter  20 value 208.659185
## iter  30 value 202.464051
## iter  40 value 202.104437
## iter  50 value 201.291682
## iter  60 value 201.049826
## iter  70 value 201.046732
## final  value 201.045468 
## converged
## # weights:  64
## initial  value 555.870979 
## iter  10 value 192.524036
## iter  20 value 126.570538
## iter  30 value 106.338919
## iter  40 value 97.636856
## iter  50 value 91.462720
## iter  60 value 89.715380
## iter  70 value 88.588613
## iter  80 value 87.982672
## iter  90 value 87.629137
## iter 100 value 87.326022
## final  value 87.326022 
## stopped after 100 iterations
## # weights:  106
## initial  value 472.986693 
## iter  10 value 173.157290
## iter  20 value 127.529338
## iter  30 value 94.806812
## iter  40 value 88.287478
## iter  50 value 85.014470
## iter  60 value 78.201744
## iter  70 value 75.229542
## iter  80 value 71.268208
## iter  90 value 67.327955
## iter 100 value 67.003390
## final  value 67.003390 
## stopped after 100 iterations
## # weights:  22
## initial  value 503.835036 
## iter  10 value 258.598679
## iter  20 value 223.000398
## iter  30 value 211.960998
## iter  40 value 197.273722
## iter  50 value 192.746363
## iter  60 value 190.973027
## iter  70 value 189.082588
## iter  80 value 188.687205
## iter  90 value 186.036415
## iter 100 value 185.778627
## final  value 185.778627 
## stopped after 100 iterations
## # weights:  64
## initial  value 468.382035 
## iter  10 value 203.574098
## iter  20 value 167.598964
## iter  30 value 137.278794
## iter  40 value 124.329418
## iter  50 value 118.944769
## iter  60 value 115.687461
## iter  70 value 114.348110
## iter  80 value 113.156840
## iter  90 value 112.663068
## iter 100 value 112.388184
## final  value 112.388184 
## stopped after 100 iterations
## # weights:  106
## initial  value 471.278766 
## iter  10 value 217.119363
## iter  20 value 121.352401
## iter  30 value 96.024085
## iter  40 value 88.022803
## iter  50 value 83.726900
## iter  60 value 79.923680
## iter  70 value 74.811410
## iter  80 value 74.024481
## iter  90 value 61.019476
## iter 100 value 56.514372
## final  value 56.514372 
## stopped after 100 iterations
## # weights:  22
## initial  value 491.556999 
## iter  10 value 241.729292
## iter  20 value 226.190241
## iter  30 value 222.001952
## iter  40 value 221.802029
## final  value 221.802024 
## converged
## # weights:  64
## initial  value 520.594948 
## iter  10 value 214.924839
## iter  20 value 179.595307
## iter  30 value 171.067742
## iter  40 value 169.715990
## iter  50 value 169.284259
## iter  60 value 168.981097
## iter  70 value 168.922896
## iter  80 value 168.902398
## final  value 168.902050 
## converged
## # weights:  106
## initial  value 465.416170 
## iter  10 value 191.494642
## iter  20 value 150.049109
## iter  30 value 134.291123
## iter  40 value 129.056194
## iter  50 value 126.561617
## iter  60 value 125.582437
## iter  70 value 124.018671
## iter  80 value 123.681497
## iter  90 value 123.301549
## iter 100 value 122.514975
## final  value 122.514975 
## stopped after 100 iterations
## # weights:  22
## initial  value 521.388965 
## iter  10 value 222.860991
## iter  20 value 216.147391
## iter  30 value 215.065923
## final  value 215.008697 
## converged
## # weights:  64
## initial  value 464.987451 
## iter  10 value 198.189794
## iter  20 value 172.110457
## iter  30 value 157.424054
## iter  40 value 156.470198
## iter  50 value 156.342971
## iter  60 value 155.909622
## iter  70 value 155.555634
## iter  80 value 155.319836
## iter  90 value 154.857941
## iter 100 value 154.220542
## final  value 154.220542 
## stopped after 100 iterations
## # weights:  106
## initial  value 462.563701 
## iter  10 value 187.500994
## iter  20 value 114.668723
## iter  30 value 90.867648
## iter  40 value 72.098882
## iter  50 value 58.730675
## iter  60 value 56.828519
## iter  70 value 56.500690
## iter  80 value 55.561940
## iter  90 value 55.354682
## iter 100 value 55.143939
## final  value 55.143939 
## stopped after 100 iterations
## # weights:  22
## initial  value 503.073027 
## iter  10 value 266.568385
## iter  20 value 217.724263
## iter  30 value 210.089408
## iter  40 value 203.209121
## iter  50 value 201.730234
## iter  60 value 201.538905
## iter  70 value 200.502936
## iter  80 value 197.988272
## iter  90 value 197.942984
## final  value 197.942960 
## converged
## # weights:  64
## initial  value 463.225910 
## iter  10 value 211.529019
## iter  20 value 158.179824
## iter  30 value 134.759613
## iter  40 value 124.250628
## iter  50 value 118.921417
## iter  60 value 117.018413
## iter  70 value 116.330013
## iter  80 value 116.271961
## iter  90 value 116.102879
## iter 100 value 115.990479
## final  value 115.990479 
## stopped after 100 iterations
## # weights:  106
## initial  value 557.435038 
## iter  10 value 207.391040
## iter  20 value 137.433395
## iter  30 value 94.367417
## iter  40 value 70.397979
## iter  50 value 49.203502
## iter  60 value 48.217860
## iter  70 value 48.118163
## iter  80 value 48.113912
## final  value 48.113891 
## converged
## # weights:  22
## initial  value 470.589756 
## iter  10 value 257.841457
## iter  20 value 239.119308
## iter  30 value 232.787227
## iter  40 value 227.783360
## iter  50 value 221.959796
## iter  60 value 219.698254
## iter  70 value 218.723244
## iter  80 value 218.722942
## final  value 218.722936 
## converged
## # weights:  64
## initial  value 537.609918 
## iter  10 value 237.115283
## iter  20 value 194.165823
## iter  30 value 167.384159
## iter  40 value 150.930152
## iter  50 value 137.164967
## iter  60 value 133.010058
## iter  70 value 131.995469
## iter  80 value 131.761986
## iter  90 value 131.738579
## iter 100 value 131.710357
## final  value 131.710357 
## stopped after 100 iterations
## # weights:  106
## initial  value 459.652324 
## iter  10 value 190.478925
## iter  20 value 171.656328
## iter  30 value 142.269203
## iter  40 value 133.721992
## iter  50 value 129.802858
## iter  60 value 124.854983
## iter  70 value 123.988006
## iter  80 value 123.228425
## iter  90 value 123.066805
## iter 100 value 123.060794
## final  value 123.060794 
## stopped after 100 iterations
## # weights:  22
## initial  value 479.570991 
## iter  10 value 239.588741
## iter  20 value 210.838259
## iter  30 value 204.324875
## iter  40 value 202.009221
## iter  50 value 201.788798
## iter  60 value 201.765551
## iter  70 value 201.761330
## iter  80 value 201.760186
## iter  90 value 201.750955
## iter 100 value 201.750538
## final  value 201.750538 
## stopped after 100 iterations
## # weights:  64
## initial  value 600.451611 
## iter  10 value 206.318694
## iter  20 value 143.571415
## iter  30 value 128.779917
## iter  40 value 117.098451
## iter  50 value 112.326112
## iter  60 value 111.499250
## iter  70 value 111.231636
## iter  80 value 111.155016
## iter  90 value 110.703891
## iter 100 value 110.665171
## final  value 110.665171 
## stopped after 100 iterations
## # weights:  106
## initial  value 478.245264 
## iter  10 value 187.906782
## iter  20 value 139.133392
## iter  30 value 101.679058
## iter  40 value 91.434548
## iter  50 value 88.619352
## iter  60 value 86.130642
## iter  70 value 82.179961
## iter  80 value 81.317389
## iter  90 value 79.211189
## iter 100 value 78.512235
## final  value 78.512235 
## stopped after 100 iterations
## # weights:  22
## initial  value 490.624575 
## iter  10 value 222.640802
## iter  20 value 210.204129
## iter  30 value 207.842664
## final  value 207.820739 
## converged
## # weights:  64
## initial  value 533.390017 
## iter  10 value 214.539914
## iter  20 value 152.390520
## iter  30 value 133.106632
## iter  40 value 121.748357
## iter  50 value 116.226451
## iter  60 value 115.175046
## iter  70 value 114.397849
## iter  80 value 113.958598
## iter  90 value 113.837299
## iter 100 value 113.600500
## final  value 113.600500 
## stopped after 100 iterations
## # weights:  106
## initial  value 629.919946 
## iter  10 value 185.675646
## iter  20 value 125.621372
## iter  30 value 96.091136
## iter  40 value 87.031036
## iter  50 value 82.748751
## iter  60 value 80.227050
## iter  70 value 79.326094
## iter  80 value 78.638379
## iter  90 value 77.723398
## iter 100 value 77.123577
## final  value 77.123577 
## stopped after 100 iterations
## # weights:  22
## initial  value 533.279188 
## iter  10 value 225.101048
## iter  20 value 219.312344
## iter  30 value 214.636135
## iter  40 value 213.943194
## final  value 213.942827 
## converged
## # weights:  64
## initial  value 508.191424 
## iter  10 value 233.604479
## iter  20 value 206.964656
## iter  30 value 201.713659
## iter  40 value 197.634508
## iter  50 value 185.914558
## iter  60 value 182.784930
## iter  70 value 180.769323
## iter  80 value 180.145534
## iter  90 value 180.007041
## iter 100 value 179.940679
## final  value 179.940679 
## stopped after 100 iterations
## # weights:  106
## initial  value 499.185031 
## iter  10 value 213.786829
## iter  20 value 164.428289
## iter  30 value 136.221598
## iter  40 value 123.887468
## iter  50 value 120.483084
## iter  60 value 117.818744
## iter  70 value 115.271610
## iter  80 value 112.719480
## iter  90 value 110.833728
## iter 100 value 108.899260
## final  value 108.899260 
## stopped after 100 iterations
## # weights:  22
## initial  value 538.341476 
## iter  10 value 237.496285
## iter  20 value 214.833840
## iter  30 value 211.636876
## iter  40 value 207.646113
## iter  50 value 205.566261
## iter  60 value 205.486486
## iter  70 value 205.452306
## iter  80 value 205.441142
## iter  90 value 205.430359
## iter 100 value 205.424095
## final  value 205.424095 
## stopped after 100 iterations
## # weights:  64
## initial  value 546.609356 
## iter  10 value 196.046443
## iter  20 value 163.379190
## iter  30 value 150.941530
## iter  40 value 139.989574
## iter  50 value 132.464421
## iter  60 value 131.689669
## iter  70 value 131.417755
## iter  80 value 131.168936
## iter  90 value 130.905867
## iter 100 value 130.399112
## final  value 130.399112 
## stopped after 100 iterations
## # weights:  106
## initial  value 510.598170 
## iter  10 value 192.445633
## iter  20 value 97.861480
## iter  30 value 81.337435
## iter  40 value 74.442601
## iter  50 value 71.031093
## iter  60 value 68.804470
## iter  70 value 68.133115
## iter  80 value 67.547104
## iter  90 value 66.649585
## iter 100 value 65.850290
## final  value 65.850290 
## stopped after 100 iterations
## # weights:  22
## initial  value 482.277553 
## iter  10 value 228.008550
## iter  20 value 206.821881
## iter  30 value 200.761631
## iter  40 value 197.892023
## iter  50 value 197.586067
## iter  60 value 197.577746
## iter  70 value 197.575328
## final  value 197.574563 
## converged
## # weights:  64
## initial  value 517.528887 
## iter  10 value 228.219852
## iter  20 value 155.951297
## iter  30 value 120.490872
## iter  40 value 93.209961
## iter  50 value 87.118503
## iter  60 value 85.365829
## iter  70 value 84.694504
## iter  80 value 83.771429
## iter  90 value 83.215521
## iter 100 value 83.019680
## final  value 83.019680 
## stopped after 100 iterations
## # weights:  106
## initial  value 601.950965 
## iter  10 value 215.549978
## iter  20 value 150.788992
## iter  30 value 133.393255
## iter  40 value 116.117468
## iter  50 value 108.970612
## iter  60 value 103.296244
## iter  70 value 100.036144
## iter  80 value 98.580067
## iter  90 value 97.060140
## iter 100 value 96.544356
## final  value 96.544356 
## stopped after 100 iterations
## # weights:  22
## initial  value 478.527215 
## iter  10 value 304.898358
## iter  20 value 259.597579
## iter  30 value 228.066986
## iter  40 value 216.348298
## iter  50 value 212.528975
## iter  60 value 211.513613
## iter  70 value 211.442415
## final  value 211.442411 
## converged
## # weights:  64
## initial  value 491.497249 
## iter  10 value 281.608746
## iter  20 value 229.889456
## iter  30 value 183.816798
## iter  40 value 164.195440
## iter  50 value 154.968970
## iter  60 value 149.067140
## iter  70 value 145.639178
## iter  80 value 144.180578
## iter  90 value 143.833329
## iter 100 value 143.751144
## final  value 143.751144 
## stopped after 100 iterations
## # weights:  106
## initial  value 561.592704 
## iter  10 value 248.969846
## iter  20 value 196.311444
## iter  30 value 164.283067
## iter  40 value 138.360388
## iter  50 value 128.524126
## iter  60 value 117.399882
## iter  70 value 114.585085
## iter  80 value 113.938279
## iter  90 value 113.676978
## iter 100 value 113.497700
## final  value 113.497700 
## stopped after 100 iterations
## # weights:  22
## initial  value 504.884748 
## iter  10 value 229.078881
## iter  20 value 208.336919
## iter  30 value 205.227520
## iter  40 value 204.679952
## final  value 204.677332 
## converged
## # weights:  64
## initial  value 472.753790 
## iter  10 value 173.304453
## iter  20 value 131.657605
## iter  30 value 119.242983
## iter  40 value 106.766844
## iter  50 value 101.878823
## iter  60 value 98.834684
## iter  70 value 98.599788
## iter  80 value 98.242664
## iter  90 value 98.021181
## iter 100 value 97.949987
## final  value 97.949987 
## stopped after 100 iterations
## # weights:  106
## initial  value 571.026805 
## iter  10 value 207.663813
## iter  20 value 133.913842
## iter  30 value 83.396009
## iter  40 value 69.472277
## iter  50 value 56.934879
## iter  60 value 50.527696
## iter  70 value 49.413069
## iter  80 value 49.035627
## iter  90 value 48.202948
## iter 100 value 47.578535
## final  value 47.578535 
## stopped after 100 iterations
## # weights:  22
## initial  value 482.549408 
## iter  10 value 275.280775
## iter  20 value 215.460663
## iter  30 value 207.963343
## iter  40 value 199.940581
## iter  50 value 199.659451
## iter  60 value 199.650021
## iter  70 value 199.647067
## iter  80 value 199.646226
## iter  90 value 199.645671
## final  value 199.645579 
## converged
## # weights:  64
## initial  value 487.874948 
## iter  10 value 211.956124
## iter  20 value 173.447508
## iter  30 value 140.080236
## iter  40 value 131.029077
## iter  50 value 120.309696
## iter  60 value 115.781480
## iter  70 value 113.872838
## iter  80 value 113.496045
## iter  90 value 112.990074
## iter 100 value 112.500253
## final  value 112.500253 
## stopped after 100 iterations
## # weights:  106
## initial  value 489.480874 
## iter  10 value 224.101220
## iter  20 value 145.540325
## iter  30 value 101.524191
## iter  40 value 80.412037
## iter  50 value 74.679314
## iter  60 value 74.271843
## iter  70 value 74.266162
## final  value 74.265773 
## converged
## # weights:  22
## initial  value 479.191873 
## iter  10 value 305.277501
## iter  20 value 265.345831
## iter  30 value 249.654598
## iter  40 value 225.009267
## iter  50 value 220.045811
## iter  60 value 218.518627
## final  value 218.512131 
## converged
## # weights:  64
## initial  value 457.431317 
## iter  10 value 221.269451
## iter  20 value 176.902649
## iter  30 value 164.202151
## iter  40 value 154.836446
## iter  50 value 153.103268
## iter  60 value 152.043622
## iter  70 value 151.844812
## iter  80 value 151.818931
## final  value 151.818855 
## converged
## # weights:  106
## initial  value 508.572741 
## iter  10 value 240.174383
## iter  20 value 190.497945
## iter  30 value 164.135414
## iter  40 value 153.082995
## iter  50 value 140.871428
## iter  60 value 132.947766
## iter  70 value 128.267256
## iter  80 value 119.078645
## iter  90 value 117.614718
## iter 100 value 116.692207
## final  value 116.692207 
## stopped after 100 iterations
## # weights:  22
## initial  value 476.697854 
## iter  10 value 334.967937
## iter  20 value 282.664015
## iter  30 value 222.267958
## iter  40 value 212.981549
## iter  50 value 199.071730
## iter  60 value 195.214385
## iter  70 value 187.816529
## iter  80 value 187.742041
## iter  90 value 187.704229
## iter 100 value 187.691558
## final  value 187.691558 
## stopped after 100 iterations
## # weights:  64
## initial  value 566.504548 
## iter  10 value 299.130276
## iter  20 value 195.857937
## iter  30 value 170.559109
## iter  40 value 156.982426
## iter  50 value 145.195198
## iter  60 value 142.071571
## iter  70 value 141.559744
## iter  80 value 140.008499
## iter  90 value 139.206068
## iter 100 value 137.810607
## final  value 137.810607 
## stopped after 100 iterations
## # weights:  106
## initial  value 550.878011 
## iter  10 value 184.486618
## iter  20 value 121.682562
## iter  30 value 94.252018
## iter  40 value 71.277466
## iter  50 value 66.661087
## iter  60 value 64.965166
## iter  70 value 63.606443
## iter  80 value 61.708375
## iter  90 value 61.138278
## iter 100 value 60.379710
## final  value 60.379710 
## stopped after 100 iterations
## # weights:  22
## initial  value 496.590281 
## iter  10 value 241.469850
## iter  20 value 209.832480
## iter  30 value 206.065956
## iter  40 value 195.006639
## iter  50 value 194.795961
## iter  60 value 194.794110
## iter  70 value 194.792551
## final  value 194.792126 
## converged
## # weights:  64
## initial  value 589.639089 
## iter  10 value 201.668448
## iter  20 value 143.893677
## iter  30 value 111.524107
## iter  40 value 96.684742
## iter  50 value 87.518577
## iter  60 value 80.561443
## iter  70 value 70.783637
## iter  80 value 68.451110
## iter  90 value 66.329588
## iter 100 value 66.256496
## final  value 66.256496 
## stopped after 100 iterations
## # weights:  106
## initial  value 521.704818 
## iter  10 value 193.685918
## iter  20 value 137.947185
## iter  30 value 100.709722
## iter  40 value 67.983825
## iter  50 value 58.484890
## iter  60 value 55.499838
## iter  70 value 52.144689
## iter  80 value 51.700068
## iter  90 value 51.419255
## iter 100 value 51.310597
## final  value 51.310597 
## stopped after 100 iterations
## # weights:  22
## initial  value 474.611216 
## iter  10 value 253.598154
## iter  20 value 224.535685
## iter  30 value 217.398120
## final  value 217.393113 
## converged
## # weights:  64
## initial  value 483.778745 
## iter  10 value 200.896445
## iter  20 value 157.502901
## iter  30 value 150.192851
## iter  40 value 147.801373
## iter  50 value 147.007784
## iter  60 value 146.797344
## iter  70 value 146.767961
## final  value 146.767079 
## converged
## # weights:  106
## initial  value 478.671866 
## iter  10 value 189.861550
## iter  20 value 143.557450
## iter  30 value 133.184573
## iter  40 value 126.180046
## iter  50 value 124.318961
## iter  60 value 123.155455
## iter  70 value 122.970030
## iter  80 value 122.916477
## iter  90 value 122.224699
## iter 100 value 121.692491
## final  value 121.692491 
## stopped after 100 iterations
## # weights:  22
## initial  value 496.045000 
## iter  10 value 281.283588
## iter  20 value 250.801994
## iter  30 value 217.136551
## iter  40 value 210.466873
## iter  50 value 206.798219
## iter  60 value 204.728980
## iter  70 value 202.882020
## iter  80 value 202.845438
## iter  90 value 202.825884
## iter 100 value 202.824628
## final  value 202.824628 
## stopped after 100 iterations
## # weights:  64
## initial  value 506.447764 
## iter  10 value 227.987444
## iter  20 value 168.497994
## iter  30 value 153.622286
## iter  40 value 138.335234
## iter  50 value 126.665422
## iter  60 value 121.527205
## iter  70 value 119.696532
## iter  80 value 119.178456
## iter  90 value 118.962614
## iter 100 value 117.359415
## final  value 117.359415 
## stopped after 100 iterations
## # weights:  106
## initial  value 463.503059 
## iter  10 value 211.316842
## iter  20 value 125.595613
## iter  30 value 96.120852
## iter  40 value 82.929951
## iter  50 value 80.176199
## iter  60 value 79.352062
## iter  70 value 78.792841
## iter  80 value 78.327423
## iter  90 value 77.452530
## iter 100 value 76.822641
## final  value 76.822641 
## stopped after 100 iterations
## # weights:  106
## initial  value 585.930121 
## iter  10 value 202.208353
## iter  20 value 136.327012
## iter  30 value 96.791432
## iter  40 value 66.134100
## iter  50 value 48.497589
## iter  60 value 45.980674
## iter  70 value 45.212678
## iter  80 value 45.208154
## iter  90 value 45.208076
## final  value 45.208076 
## converged

Auffallend war, dass dieses Modell länger gerechnet hat, dies hängt vermutlich damit zusammen, dass es durch das Dummy Encoden mehr Inputvariablen zum Verarbeiten hat.

plot(modell_nn5)

Die beiden neuronalen Netze mit der Train Function von Caret performen sehr ähnlich. Auch der beste Fit von den Parametern identisch. Der einzige Unterschied liegt darin, dass die Modelle mit geringeren Weights beim Modell mit One Hot Encodeden Daten und upgesamplten Trainingsset bei mehr Hidden Units besser performen. Aber der Unterschied ist im dezimalen Prozentbereich.

modell_nn5_best <- modell_nn5$bestTune
modell_nn5_best

##   size decay
## 7    5     0

predict_testNN_5 = predict(modell_nn5, testset_nn)
#predict_testNN_5 <-sapply(predict_testNN_5,round,digits=0)
nn_table5 <- table(testset_nn$target, predict_testNN_5)

results_nn5 <- data.frame(actual = testset_nn$target, prediction = predict_testNN_5)
conf_nn5 <- confusionMatrix(nn_table5)
conf_nn5

## Confusion Matrix and Statistics
## 
##    predict_testNN_5
##      0  1
##   0 74 20
##   1  2  9
##                                           
##                Accuracy : 0.7905          
##                  95% CI : (0.7001, 0.8638)
##     No Information Rate : 0.7238          
##     P-Value [Acc > NIR] : 0.0751910       
##                                           
##                   Kappa : 0.3515          
##                                           
##  Mcnemar's Test P-Value : 0.0002896       
##                                           
##             Sensitivity : 0.9737          
##             Specificity : 0.3103          
##          Pos Pred Value : 0.7872          
##          Neg Pred Value : 0.8182          
##              Prevalence : 0.7238          
##          Detection Rate : 0.7048          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.6420          
##                                           
##        'Positive' Class : 0               
## 

Dieses Modell kommt auf eine Testaccuracy von 79 %. Auffällig ist, dass dieses Modell 20 Patient als Corona Infiziert ausgeben, obwohl die Patienten gesund sind. Wobei lediglich 2 Patienten fälschlicherweise als gesund ausgegeben werden. Die Specificity ist trotzdem sehr schwach mit nur etwas über 31 %.

acc_nn5 <- conf_nn5$overall[1]
sens_nn5 <- conf_nn5$byClass[1]
spec_nn5 <- conf_nn5$byClass[2]

In der Übersicht erkennt man sehr deutlich, dass die beiden Neuronale Netze mit der nnet Function (einmal auf Skalierten und Encodeden Daten und einmal auf nicht vorverarbeiten) sehr ähnlich abschneiden, mit einer TestAccuracy von ca. 75%. Aber für die Vorhersage von tatsächlich Corona Infizierten sind diese beiden Modelle nicht nützlich, da Sie nur eine Specificity von 25% aufweisen. Das beiden Neuronalen Netze mit der Caret Library hingegen kommt auf eine Accuracy von 100% und sagt samit alle 289 Patienten im Testdatensatz korrekt voraus. Das Netz wurde mit 10 Fold Cross Validation und 3 facher Wiederholhung trainiert. Die Daten wurden zu dem min-max skaliert.

library(kableExtra)
modell <- c(2,3,4,5)
test_acc <- c(acc_nn2, acc_nn3, acc_nn4, acc_nn5)
sens <- c(sens_nn2, sens_nn3, sens_nn4, sens_nn5)
spec <- c(spec_nn2, spec_nn3, spec_nn4, spec_nn5)
results_nn = data.frame(
  "model" = modell,
  "sensitivity" = sens,
  "Specificity" = spec,
  "Test Accuracy" = test_acc
)

kable_styling(kable(results_nn, format = "html", digits = 4), full_width = FALSE)

model	sensitivity	Specificity	Test.Accuracy
2	0.9231	0.1852	0.7333
3	0.9643	0.3810	0.8476
4	0.9474	0.6000	0.9143
5	0.9737	0.3103	0.7905

train_eng_nn <- read.csv("data/clean/train_feat_eng.csv")
test_eng_nn <- read.csv("data/clean/test_feat_eng.csv")

glimpse(train_eng_nn)

## Rows: 760
## Columns: 16
## $ age               <int> 17, 1, 9, 11, 13, 9, 17, 17, 19, 10, 11, 11, 16, 15…
## $ target            <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ reg_ward          <int> 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ semi_unit         <int> 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, …
## $ intense_unit      <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, …
## $ sickness          <int> 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, …
## $ Hematocrit        <dbl> 0.23651545, -1.57168221, -0.74769306, 0.99183822, 1…
## $ Platelets         <dbl> -0.51741302, 1.42966747, -0.42948034, 0.07299204, -…
## $ Platelets_vol     <dbl> 0.01067657, -1.67222178, -0.21371073, -0.55028951, …
## $ Lymphocytes       <dbl> 0.318365753, -0.005738043, -1.114513755, 0.04543625…
## $ mean_hemoglobin   <dbl> -0.95079035, 3.33107066, 0.54288238, -0.45289949, -…
## $ Leukocytes        <dbl> -9.461035e-02, 3.645505e-01, -8.849232e-01, -2.1148…
## $ Eosinophils       <dbl> 1.48215818, 1.01862502, -0.66695017, -0.70908952, 0…
## $ Monocytes         <dbl> 0.35754666, 0.06865151, 1.27675891, -0.22024387, 0.…
## $ age_plat_leuk_eos <dbl> 19.401150, 5.945848, 18.735579, 17.303070, 17.41957…
## $ age_leuk_eos      <dbl> 10.059854, 4.281499, 9.859892, 9.911350, 9.564118, …

set.seed(1910837262)
up_train_eng_nn <- upSample(x = train_eng_nn[, -ncol(train_eng_nn)],
                     y = as.factor(train_eng_nn$target))                        
table(up_train_eng_nn$target) 

## 
##   0   1 
## 380 380

up_train_eng_nn <- up_train_eng_nn %>%
  select(-Class)

up_train_eng_nn_x <- up_train_eng_nn %>%
  select(-target)

Nun trainieren wir noch ein Neuronales Netz mit den Inputdaten, die wir durch das Feature Engineering ein wenig verändert haben. Die weiteren Prozessschritte lassen wir aber identisch.

modell_nn6 <- train(up_train_eng_nn[,-2], up_train_eng_nn$target,
                  method = "nnet",
                  trControl= TrainingParameters_nn,
                  preProcess=c("scale","center"),
                  na.action = na.omit
)

## Warning in train.default(up_train_eng_nn[, -2], up_train_eng_nn$target, : You
## are trying to do regression and your outcome only has two possible values Are
## you trying to do classification? If so, use a 2 level factor as your outcome
## column.

## # weights:  17
## initial  value 171.345733 
## iter  10 value 128.000002
## iter  10 value 128.000001
## iter  10 value 128.000001
## final  value 128.000001 
## converged
## # weights:  49
## initial  value 201.210288 
## iter  10 value 104.044493
## iter  20 value 96.693825
## iter  30 value 95.692730
## iter  40 value 95.691431
## iter  50 value 94.665053
## iter  60 value 88.326844
## iter  70 value 84.656432
## iter  80 value 84.027029
## iter  90 value 83.905825
## iter 100 value 83.026560
## final  value 83.026560 
## stopped after 100 iterations
## # weights:  81
## initial  value 178.180649 
## iter  10 value 91.352462
## iter  20 value 83.003524
## iter  30 value 82.996752
## iter  40 value 82.995457
## iter  50 value 82.988790
## iter  60 value 80.798693
## iter  70 value 78.831166
## iter  80 value 78.798153
## iter  90 value 78.632406
## iter 100 value 76.802411
## final  value 76.802411 
## stopped after 100 iterations
## # weights:  17
## initial  value 177.104368 
## iter  10 value 72.656883
## iter  20 value 63.677760
## iter  30 value 63.033007
## final  value 63.025587 
## converged
## # weights:  49
## initial  value 174.620971 
## iter  10 value 91.643069
## iter  20 value 62.158502
## iter  30 value 56.348794
## iter  40 value 55.136977
## iter  50 value 54.662901
## iter  60 value 50.003806
## iter  70 value 49.482208
## iter  80 value 49.430187
## final  value 49.429809 
## converged
## # weights:  81
## initial  value 217.427529 
## iter  10 value 90.687721
## iter  20 value 56.010953
## iter  30 value 46.923848
## iter  40 value 46.060359
## iter  50 value 45.741460
## iter  60 value 45.369228
## iter  70 value 44.980336
## iter  80 value 44.288574
## iter  90 value 43.267286
## iter 100 value 42.284157
## final  value 42.284157 
## stopped after 100 iterations
## # weights:  17
## initial  value 203.352562 
## iter  10 value 100.273876
## iter  20 value 93.023094
## iter  30 value 77.991189
## iter  40 value 72.012273
## iter  50 value 70.099354
## iter  60 value 70.087049
## iter  70 value 70.083957
## iter  80 value 69.889246
## iter  90 value 68.963301
## iter 100 value 61.783012
## final  value 61.783012 
## stopped after 100 iterations
## # weights:  49
## initial  value 157.127268 
## iter  10 value 120.639996
## iter  20 value 114.571690
## iter  30 value 109.380720
## iter  40 value 95.476896
## iter  50 value 87.334573
## iter  60 value 80.549543
## iter  70 value 78.911834
## iter  80 value 77.456415
## iter  90 value 77.047322
## iter 100 value 76.094057
## final  value 76.094057 
## stopped after 100 iterations
## # weights:  81
## initial  value 185.767949 
## iter  10 value 125.914183
## iter  20 value 113.715865
## iter  30 value 97.705113
## iter  40 value 94.745816
## iter  50 value 94.742373
## iter  60 value 94.738079
## iter  70 value 94.732598
## iter  80 value 94.726497
## iter  90 value 94.718500
## iter 100 value 94.675853
## final  value 94.675853 
## stopped after 100 iterations
## # weights:  17
## initial  value 170.680068 
## iter  10 value 66.347317
## iter  20 value 56.938466
## iter  30 value 56.477297
## iter  40 value 54.800785
## iter  50 value 54.596861
## iter  60 value 54.534954
## iter  70 value 54.474787
## iter  80 value 54.438047
## iter  90 value 54.429811
## iter 100 value 54.416545
## final  value 54.416545 
## stopped after 100 iterations
## # weights:  49
## initial  value 194.654794 
## iter  10 value 107.000905
## iter  20 value 107.000001
## iter  20 value 107.000000
## iter  20 value 107.000000
## final  value 107.000000 
## converged
## # weights:  81
## initial  value 172.967315 
## iter  10 value 105.999991
## iter  20 value 105.999436
## iter  30 value 100.588325
## iter  40 value 87.977004
## iter  50 value 81.778893
## iter  60 value 77.118054
## iter  70 value 77.003350
## iter  80 value 77.000773
## iter  90 value 77.000413
## iter 100 value 77.000122
## final  value 77.000122 
## stopped after 100 iterations
## # weights:  17
## initial  value 173.520568 
## iter  10 value 118.496851
## iter  20 value 71.683496
## iter  30 value 63.642325
## iter  40 value 63.517734
## iter  50 value 63.498360
## iter  60 value 63.480293
## final  value 63.480261 
## converged
## # weights:  49
## initial  value 184.450994 
## iter  10 value 69.484136
## iter  20 value 56.927251
## iter  30 value 53.961280
## iter  40 value 52.328670
## iter  50 value 51.884202
## iter  60 value 51.851214
## iter  70 value 51.847639
## final  value 51.847614 
## converged
## # weights:  81
## initial  value 222.308002 
## iter  10 value 84.266596
## iter  20 value 59.096787
## iter  30 value 55.407557
## iter  40 value 52.004365
## iter  50 value 50.028585
## iter  60 value 49.305397
## iter  70 value 48.876715
## iter  80 value 48.790703
## iter  90 value 48.721597
## iter 100 value 48.615977
## final  value 48.615977 
## stopped after 100 iterations
## # weights:  17
## initial  value 175.083922 
## iter  10 value 84.744334
## iter  20 value 73.121407
## iter  30 value 72.762641
## iter  40 value 72.668407
## iter  50 value 72.643737
## iter  60 value 72.641143
## iter  70 value 72.639337
## iter  80 value 72.638289
## iter  90 value 72.636337
## iter 100 value 72.635536
## final  value 72.635536 
## stopped after 100 iterations
## # weights:  49
## initial  value 183.852397 
## iter  10 value 118.810689
## iter  20 value 107.107331
## iter  30 value 83.075698
## iter  40 value 59.016393
## iter  50 value 55.727656
## iter  60 value 54.553524
## iter  70 value 53.509503
## iter  80 value 52.053692
## iter  90 value 50.869502
## iter 100 value 49.575288
## final  value 49.575288 
## stopped after 100 iterations
## # weights:  81
## initial  value 208.988793 
## iter  10 value 91.301449
## iter  20 value 91.189707
## iter  30 value 89.558909
## iter  40 value 79.678487
## iter  50 value 71.245587
## iter  60 value 63.315387
## iter  70 value 61.191882
## iter  80 value 61.160502
## iter  90 value 61.113563
## iter 100 value 49.470558
## final  value 49.470558 
## stopped after 100 iterations
## # weights:  17
## initial  value 173.302807 
## iter  10 value 103.662173
## iter  20 value 93.887900
## iter  30 value 87.046635
## iter  40 value 84.005627
## iter  50 value 83.653506
## iter  60 value 81.730767
## iter  70 value 80.066134
## iter  80 value 79.994609
## iter  90 value 79.982844
## iter 100 value 73.867214
## final  value 73.867214 
## stopped after 100 iterations
## # weights:  49
## initial  value 222.601520 
## iter  10 value 91.581922
## iter  20 value 84.028087
## iter  30 value 75.109033
## iter  40 value 52.064389
## iter  50 value 46.222759
## iter  60 value 45.528408
## iter  70 value 44.497497
## iter  80 value 44.449914
## iter  90 value 44.423158
## iter 100 value 44.390987
## final  value 44.390987 
## stopped after 100 iterations
## # weights:  81
## initial  value 186.069752 
## iter  10 value 101.168491
## iter  20 value 89.429545
## iter  30 value 77.962745
## iter  40 value 76.825853
## iter  50 value 76.803404
## iter  60 value 76.800433
## iter  70 value 76.800114
## final  value 76.800072 
## converged
## # weights:  17
## initial  value 177.127920 
## iter  10 value 98.130900
## iter  20 value 73.351621
## iter  30 value 65.025879
## iter  40 value 63.039958
## iter  50 value 63.025267
## final  value 63.025135 
## converged
## # weights:  49
## initial  value 177.357009 
## iter  10 value 66.597480
## iter  20 value 55.977352
## iter  30 value 53.814651
## iter  40 value 53.446205
## iter  50 value 53.261386
## iter  60 value 53.257721
## final  value 53.257701 
## converged
## # weights:  81
## initial  value 187.161465 
## iter  10 value 71.889059
## iter  20 value 62.389248
## iter  30 value 56.311374
## iter  40 value 52.984925
## iter  50 value 51.744373
## iter  60 value 51.349058
## iter  70 value 51.166127
## iter  80 value 50.950193
## iter  90 value 50.867514
## iter 100 value 50.780376
## final  value 50.780376 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.772837 
## iter  10 value 118.772193
## iter  20 value 118.752858
## iter  30 value 118.661424
## iter  40 value 106.892119
## iter  50 value 103.964610
## iter  60 value 99.053701
## iter  70 value 98.012839
## iter  80 value 97.032227
## iter  90 value 93.376498
## iter 100 value 92.021195
## final  value 92.021195 
## stopped after 100 iterations
## # weights:  49
## initial  value 179.078192 
## iter  10 value 56.363085
## iter  20 value 40.289703
## iter  30 value 34.323467
## iter  40 value 32.273890
## iter  50 value 31.580798
## iter  60 value 31.470410
## iter  70 value 31.453693
## iter  80 value 31.432592
## iter  90 value 31.425683
## iter 100 value 31.393969
## final  value 31.393969 
## stopped after 100 iterations
## # weights:  81
## initial  value 203.092903 
## iter  10 value 105.157002
## iter  20 value 103.143072
## iter  30 value 102.341710
## iter  40 value 102.092306
## iter  50 value 55.268431
## iter  60 value 34.502158
## iter  70 value 29.141467
## iter  80 value 27.852603
## iter  90 value 27.263602
## iter 100 value 27.174666
## final  value 27.174666 
## stopped after 100 iterations
## # weights:  17
## initial  value 176.334956 
## iter  10 value 91.624714
## iter  20 value 90.969689
## iter  30 value 90.347167
## iter  40 value 87.351068
## iter  50 value 84.787363
## iter  60 value 84.135988
## iter  70 value 81.483159
## iter  80 value 81.257667
## iter  90 value 78.457604
## iter 100 value 75.356132
## final  value 75.356132 
## stopped after 100 iterations
## # weights:  49
## initial  value 152.253658 
## iter  10 value 107.938174
## iter  20 value 92.739698
## iter  30 value 91.889610
## iter  40 value 88.765634
## iter  50 value 84.923828
## iter  60 value 81.772462
## iter  70 value 80.264264
## iter  80 value 78.631665
## iter  90 value 75.356384
## iter 100 value 74.694613
## final  value 74.694613 
## stopped after 100 iterations
## # weights:  81
## initial  value 170.710025 
## iter  10 value 66.236924
## iter  20 value 55.279367
## iter  30 value 49.094185
## iter  40 value 47.951951
## iter  50 value 47.857757
## iter  60 value 47.849159
## iter  70 value 47.847769
## iter  80 value 47.834037
## iter  90 value 46.959750
## iter 100 value 46.856982
## final  value 46.856982 
## stopped after 100 iterations
## # weights:  17
## initial  value 189.597818 
## iter  10 value 106.136257
## iter  20 value 69.672426
## iter  30 value 61.830505
## iter  40 value 61.414658
## final  value 61.414356 
## converged
## # weights:  49
## initial  value 196.955128 
## iter  10 value 68.443172
## iter  20 value 57.670551
## iter  30 value 54.130447
## iter  40 value 52.388130
## iter  50 value 52.081329
## iter  60 value 50.360023
## iter  70 value 47.712259
## iter  80 value 47.360802
## iter  90 value 47.186792
## iter 100 value 47.181598
## final  value 47.181598 
## stopped after 100 iterations
## # weights:  81
## initial  value 152.574975 
## iter  10 value 98.026400
## iter  20 value 63.037002
## iter  30 value 50.526131
## iter  40 value 48.613876
## iter  50 value 47.301291
## iter  60 value 47.186663
## iter  70 value 47.175209
## iter  80 value 47.174849
## final  value 47.174845 
## converged
## # weights:  17
## initial  value 168.879285 
## iter  10 value 59.946522
## iter  20 value 54.922453
## iter  30 value 54.001010
## iter  40 value 51.844637
## iter  50 value 50.891587
## iter  60 value 50.153522
## iter  70 value 49.948202
## iter  80 value 49.907475
## iter  90 value 49.865884
## iter 100 value 49.856250
## final  value 49.856250 
## stopped after 100 iterations
## # weights:  49
## initial  value 176.223583 
## iter  10 value 94.634748
## iter  20 value 57.831728
## iter  30 value 43.704062
## iter  40 value 37.478511
## iter  50 value 28.135399
## iter  60 value 26.493888
## iter  70 value 25.981469
## iter  80 value 25.928826
## iter  90 value 25.902190
## iter 100 value 25.842225
## final  value 25.842225 
## stopped after 100 iterations
## # weights:  81
## initial  value 166.040886 
## iter  10 value 72.152590
## iter  20 value 35.529710
## iter  30 value 27.756188
## iter  40 value 27.604830
## iter  50 value 27.539129
## iter  60 value 27.473810
## iter  70 value 27.382993
## iter  80 value 27.341956
## iter  90 value 27.280554
## iter 100 value 27.251403
## final  value 27.251403 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.406275 
## iter  10 value 91.472234
## iter  20 value 68.092321
## iter  30 value 65.573470
## iter  40 value 65.469557
## iter  50 value 65.467542
## iter  60 value 65.237227
## iter  70 value 65.197152
## iter  80 value 65.194047
## iter  90 value 65.193121
## iter 100 value 65.191443
## final  value 65.191443 
## stopped after 100 iterations
## # weights:  49
## initial  value 173.832255 
## iter  10 value 102.867686
## iter  20 value 94.842771
## iter  30 value 83.292224
## iter  40 value 79.007441
## iter  50 value 74.944539
## iter  60 value 73.900259
## iter  70 value 73.625596
## iter  80 value 71.403427
## iter  90 value 70.237087
## iter 100 value 69.515921
## final  value 69.515921 
## stopped after 100 iterations
## # weights:  81
## initial  value 199.279806 
## final  value 104.999997 
## converged
## # weights:  17
## initial  value 189.926502 
## iter  10 value 70.468246
## iter  20 value 67.044830
## iter  30 value 66.866360
## final  value 66.866274 
## converged
## # weights:  49
## initial  value 184.954339 
## iter  10 value 113.681615
## iter  20 value 68.690243
## iter  30 value 57.843124
## iter  40 value 50.179727
## iter  50 value 49.614172
## iter  60 value 49.610569
## final  value 49.610529 
## converged
## # weights:  81
## initial  value 189.507656 
## iter  10 value 75.798488
## iter  20 value 59.261751
## iter  30 value 54.058081
## iter  40 value 49.309792
## iter  50 value 48.778302
## iter  60 value 48.600482
## iter  70 value 47.973862
## iter  80 value 46.838094
## iter  90 value 46.611857
## iter 100 value 46.593694
## final  value 46.593694 
## stopped after 100 iterations
## # weights:  17
## initial  value 177.490491 
## iter  10 value 102.616713
## iter  20 value 100.380956
## iter  30 value 99.457737
## iter  40 value 98.451057
## iter  50 value 97.674395
## iter  60 value 97.170133
## iter  70 value 95.094259
## iter  80 value 91.012425
## iter  90 value 90.951998
## iter 100 value 90.916647
## final  value 90.916647 
## stopped after 100 iterations
## # weights:  49
## initial  value 177.393665 
## iter  10 value 92.095866
## iter  20 value 85.252777
## iter  30 value 78.457606
## iter  40 value 78.261732
## iter  50 value 78.241288
## iter  60 value 77.230266
## iter  70 value 77.204002
## iter  80 value 76.438122
## iter  90 value 76.160275
## iter 100 value 72.734113
## final  value 72.734113 
## stopped after 100 iterations
## # weights:  81
## initial  value 213.923624 
## iter  10 value 80.753844
## iter  20 value 72.925409
## iter  30 value 71.232586
## iter  40 value 70.262325
## iter  50 value 70.235997
## iter  60 value 65.689836
## iter  70 value 63.198826
## iter  80 value 61.161191
## iter  90 value 60.250233
## iter 100 value 59.638469
## final  value 59.638469 
## stopped after 100 iterations
## # weights:  17
## initial  value 170.756502 
## iter  10 value 100.697024
## iter  20 value 95.494263
## iter  30 value 92.413857
## iter  40 value 91.470080
## iter  50 value 89.465076
## iter  60 value 87.854469
## iter  70 value 87.452769
## iter  80 value 87.422793
## iter  90 value 86.624286
## iter 100 value 86.592292
## final  value 86.592292 
## stopped after 100 iterations
## # weights:  49
## initial  value 195.499587 
## final  value 122.000000 
## converged
## # weights:  81
## initial  value 173.235721 
## iter  10 value 100.802085
## iter  20 value 97.812203
## iter  30 value 97.800041
## final  value 97.800007 
## converged
## # weights:  17
## initial  value 176.316827 
## iter  10 value 88.719571
## iter  20 value 61.678287
## iter  30 value 60.197140
## final  value 60.172412 
## converged
## # weights:  49
## initial  value 198.137834 
## iter  10 value 71.151561
## iter  20 value 55.947905
## iter  30 value 52.881260
## iter  40 value 51.847493
## iter  50 value 49.631376
## iter  60 value 48.150734
## iter  70 value 47.986085
## iter  80 value 47.967279
## iter  90 value 47.966529
## iter 100 value 47.966387
## final  value 47.966387 
## stopped after 100 iterations
## # weights:  81
## initial  value 187.926697 
## iter  10 value 93.403717
## iter  20 value 58.482268
## iter  30 value 53.635058
## iter  40 value 51.413249
## iter  50 value 49.251021
## iter  60 value 47.706025
## iter  70 value 46.067377
## iter  80 value 45.461595
## iter  90 value 45.367933
## iter 100 value 45.355548
## final  value 45.355548 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.652612 
## iter  10 value 106.486570
## iter  20 value 89.866912
## iter  30 value 87.393994
## iter  40 value 78.999863
## iter  50 value 75.892381
## iter  60 value 74.285870
## iter  70 value 73.491544
## iter  80 value 73.447788
## iter  90 value 72.672735
## iter 100 value 71.301577
## final  value 71.301577 
## stopped after 100 iterations
## # weights:  49
## initial  value 170.830212 
## iter  10 value 121.652668
## iter  20 value 70.090868
## iter  30 value 41.403236
## iter  40 value 37.310784
## iter  50 value 35.067607
## iter  60 value 32.472690
## iter  70 value 32.300367
## iter  80 value 32.203988
## iter  90 value 32.102550
## iter 100 value 31.106709
## final  value 31.106709 
## stopped after 100 iterations
## # weights:  81
## initial  value 217.967477 
## iter  10 value 101.347821
## iter  20 value 100.201263
## iter  30 value 90.031871
## iter  40 value 87.265155
## iter  50 value 82.276031
## iter  60 value 82.274613
## iter  70 value 82.271607
## iter  80 value 81.264988
## iter  90 value 81.263893
## iter 100 value 81.262777
## final  value 81.262777 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.043550 
## iter  10 value 92.050205
## iter  20 value 82.199706
## iter  30 value 69.827590
## iter  40 value 68.410670
## iter  50 value 67.910280
## iter  60 value 67.879931
## final  value 67.878473 
## converged
## # weights:  49
## initial  value 184.467038 
## iter  10 value 94.923409
## iter  20 value 85.249070
## iter  30 value 81.238634
## iter  40 value 73.845199
## iter  50 value 63.240741
## iter  60 value 61.751479
## iter  70 value 61.383496
## iter  80 value 60.843936
## iter  90 value 60.340420
## iter 100 value 60.271749
## final  value 60.271749 
## stopped after 100 iterations
## # weights:  81
## initial  value 175.965355 
## iter  10 value 80.157047
## iter  20 value 76.622530
## iter  30 value 70.924151
## iter  40 value 70.006848
## iter  50 value 69.902970
## iter  60 value 68.057271
## iter  70 value 66.069343
## iter  80 value 65.978009
## iter  90 value 65.064287
## iter 100 value 64.052600
## final  value 64.052600 
## stopped after 100 iterations
## # weights:  17
## initial  value 179.747257 
## iter  10 value 91.357432
## iter  20 value 67.279188
## iter  30 value 66.381354
## final  value 66.376991 
## converged
## # weights:  49
## initial  value 179.845541 
## iter  10 value 72.499895
## iter  20 value 52.876214
## iter  30 value 50.247971
## iter  40 value 49.645158
## iter  50 value 49.456885
## iter  60 value 49.307581
## iter  70 value 49.299517
## final  value 49.299138 
## converged
## # weights:  81
## initial  value 182.111708 
## iter  10 value 83.238362
## iter  20 value 60.841817
## iter  30 value 55.078277
## iter  40 value 53.005311
## iter  50 value 51.489390
## iter  60 value 48.178213
## iter  70 value 46.266772
## iter  80 value 45.572374
## iter  90 value 45.093840
## iter 100 value 44.928443
## final  value 44.928443 
## stopped after 100 iterations
## # weights:  17
## initial  value 173.735924 
## iter  10 value 83.758946
## iter  20 value 74.544342
## iter  30 value 66.225768
## iter  40 value 65.364924
## iter  50 value 64.098492
## iter  60 value 63.312142
## iter  70 value 63.149119
## iter  80 value 62.325494
## iter  90 value 62.176753
## iter 100 value 62.167876
## final  value 62.167876 
## stopped after 100 iterations
## # weights:  49
## initial  value 178.713316 
## iter  10 value 102.319901
## iter  20 value 101.371949
## iter  30 value 101.300344
## iter  40 value 89.690920
## iter  50 value 84.626439
## iter  60 value 83.255953
## iter  70 value 81.289013
## iter  80 value 77.268498
## iter  90 value 77.242859
## iter 100 value 77.225969
## final  value 77.225969 
## stopped after 100 iterations
## # weights:  81
## initial  value 171.333853 
## iter  10 value 115.280223
## iter  20 value 113.397705
## iter  30 value 107.982664
## iter  40 value 96.073518
## iter  50 value 85.010560
## iter  60 value 78.213028
## iter  70 value 77.062489
## iter  80 value 77.017623
## iter  90 value 76.986118
## iter 100 value 76.911716
## final  value 76.911716 
## stopped after 100 iterations
## # weights:  17
## initial  value 183.287533 
## iter  10 value 113.999573
## iter  20 value 113.000036
## final  value 113.000001 
## converged
## # weights:  49
## initial  value 196.966170 
## iter  10 value 90.897115
## iter  20 value 73.973602
## iter  30 value 71.115950
## iter  40 value 70.963200
## iter  50 value 70.080007
## iter  60 value 67.721468
## iter  70 value 66.786165
## iter  80 value 66.308313
## iter  90 value 65.864232
## iter 100 value 62.909217
## final  value 62.909217 
## stopped after 100 iterations
## # weights:  81
## initial  value 189.784649 
## final  value 101.999996 
## converged
## # weights:  17
## initial  value 177.633784 
## iter  10 value 86.456725
## iter  20 value 72.277121
## iter  30 value 64.798271
## iter  40 value 64.546452
## iter  50 value 64.541392
## final  value 64.541334 
## converged
## # weights:  49
## initial  value 161.798500 
## iter  10 value 63.472949
## iter  20 value 55.902367
## iter  30 value 54.473815
## iter  40 value 54.120739
## iter  50 value 53.707047
## iter  60 value 53.658031
## iter  70 value 53.416099
## iter  80 value 53.349029
## iter  90 value 53.347744
## final  value 53.347715 
## converged
## # weights:  81
## initial  value 177.185209 
## iter  10 value 72.924817
## iter  20 value 55.598401
## iter  30 value 51.888227
## iter  40 value 49.829847
## iter  50 value 48.748997
## iter  60 value 47.803138
## iter  70 value 46.781233
## iter  80 value 46.594446
## iter  90 value 46.560274
## iter 100 value 46.559525
## final  value 46.559525 
## stopped after 100 iterations
## # weights:  17
## initial  value 177.232157 
## iter  10 value 96.820694
## iter  20 value 81.570638
## iter  30 value 79.770573
## iter  40 value 78.837100
## iter  50 value 78.829746
## iter  60 value 78.811483
## iter  70 value 78.784679
## iter  80 value 78.770162
## iter  90 value 78.765519
## iter 100 value 78.757383
## final  value 78.757383 
## stopped after 100 iterations
## # weights:  49
## initial  value 195.610850 
## iter  10 value 116.123065
## iter  20 value 112.534994
## iter  30 value 112.213003
## iter  40 value 112.186023
## iter  50 value 110.159747
## iter  60 value 103.175461
## iter  70 value 102.339946
## iter  80 value 101.577600
## iter  90 value 100.713974
## iter 100 value 95.587448
## final  value 95.587448 
## stopped after 100 iterations
## # weights:  81
## initial  value 176.228070 
## iter  10 value 95.134723
## iter  20 value 87.332636
## iter  30 value 86.503964
## iter  40 value 86.126283
## iter  50 value 81.178301
## iter  60 value 80.056564
## iter  70 value 74.151750
## iter  80 value 73.159560
## iter  90 value 72.183846
## iter 100 value 65.584234
## final  value 65.584234 
## stopped after 100 iterations
## # weights:  17
## initial  value 181.592125 
## iter  10 value 65.478258
## iter  20 value 53.872456
## iter  30 value 51.306356
## iter  40 value 47.660762
## iter  50 value 47.005309
## iter  60 value 46.200691
## iter  70 value 45.985506
## iter  80 value 43.618121
## iter  90 value 43.545511
## final  value 43.545500 
## converged
## # weights:  49
## initial  value 181.638273 
## final  value 99.999918 
## converged
## # weights:  81
## initial  value 175.387629 
## iter  10 value 93.535400
## iter  20 value 75.785026
## iter  30 value 72.152892
## iter  40 value 70.842468
## iter  50 value 70.051946
## iter  60 value 70.000571
## iter  70 value 69.006896
## iter  80 value 69.000572
## iter  90 value 69.000139
## final  value 69.000098 
## converged
## # weights:  17
## initial  value 171.310362 
## iter  10 value 76.664273
## iter  20 value 62.670649
## iter  30 value 61.259054
## final  value 61.258087 
## converged
## # weights:  49
## initial  value 181.879817 
## iter  10 value 84.904825
## iter  20 value 61.407303
## iter  30 value 53.951516
## iter  40 value 53.219832
## iter  50 value 53.164831
## iter  60 value 53.159045
## final  value 53.158894 
## converged
## # weights:  81
## initial  value 176.619339 
## iter  10 value 56.962629
## iter  20 value 47.893927
## iter  30 value 46.636597
## iter  40 value 42.956020
## iter  50 value 41.387146
## iter  60 value 40.886678
## iter  70 value 40.707594
## iter  80 value 40.669642
## final  value 40.668402 
## converged
## # weights:  17
## initial  value 174.435524 
## iter  10 value 94.145418
## iter  20 value 80.150682
## iter  30 value 79.147483
## iter  40 value 76.146975
## iter  50 value 67.549121
## iter  60 value 63.177561
## iter  70 value 60.965526
## iter  80 value 60.304550
## iter  90 value 58.341039
## iter 100 value 58.310797
## final  value 58.310797 
## stopped after 100 iterations
## # weights:  49
## initial  value 197.439687 
## iter  10 value 94.580183
## iter  20 value 72.519063
## iter  30 value 53.091037
## iter  40 value 41.430676
## iter  50 value 39.049785
## iter  60 value 36.941079
## iter  70 value 35.842455
## iter  80 value 35.429573
## iter  90 value 34.844912
## iter 100 value 33.981722
## final  value 33.981722 
## stopped after 100 iterations
## # weights:  81
## initial  value 159.572643 
## iter  10 value 98.462078
## iter  20 value 96.580174
## iter  30 value 96.562886
## iter  40 value 96.443756
## iter  50 value 95.428170
## iter  60 value 91.552483
## iter  70 value 89.395757
## iter  80 value 88.328178
## iter  90 value 86.373612
## iter 100 value 85.322680
## final  value 85.322680 
## stopped after 100 iterations
## # weights:  17
## initial  value 167.362383 
## iter  10 value 114.812097
## iter  20 value 100.313074
## iter  30 value 92.779246
## iter  40 value 91.977263
## iter  50 value 91.906012
## iter  60 value 91.093099
## iter  70 value 91.084716
## iter  80 value 90.200040
## iter  90 value 90.174378
## iter 100 value 87.339375
## final  value 87.339375 
## stopped after 100 iterations
## # weights:  49
## initial  value 191.346277 
## iter  10 value 90.555752
## iter  20 value 78.552758
## iter  30 value 69.132893
## iter  40 value 67.049099
## iter  50 value 65.348430
## iter  60 value 59.017722
## iter  70 value 57.089016
## iter  80 value 56.611934
## iter  90 value 54.713346
## iter 100 value 53.806704
## final  value 53.806704 
## stopped after 100 iterations
## # weights:  81
## initial  value 187.959065 
## final  value 108.999984 
## converged
## # weights:  17
## initial  value 175.655779 
## iter  10 value 72.973438
## iter  20 value 66.798807
## iter  30 value 66.552608
## final  value 66.552523 
## converged
## # weights:  49
## initial  value 175.506909 
## iter  10 value 73.748432
## iter  20 value 64.988556
## iter  30 value 59.076417
## iter  40 value 56.291528
## iter  50 value 53.998818
## iter  60 value 51.335818
## iter  70 value 50.848824
## iter  80 value 50.728284
## iter  90 value 50.526037
## iter 100 value 50.491121
## final  value 50.491121 
## stopped after 100 iterations
## # weights:  81
## initial  value 161.180507 
## iter  10 value 70.654005
## iter  20 value 55.410911
## iter  30 value 50.606859
## iter  40 value 48.901079
## iter  50 value 47.533244
## iter  60 value 47.352452
## iter  70 value 47.322238
## iter  80 value 47.304868
## iter  90 value 47.302420
## iter 100 value 47.302349
## final  value 47.302349 
## stopped after 100 iterations
## # weights:  17
## initial  value 172.995610 
## iter  10 value 89.677289
## iter  20 value 82.666853
## iter  30 value 68.134974
## iter  40 value 68.004935
## iter  50 value 67.975699
## iter  60 value 67.965883
## iter  70 value 67.964213
## iter  80 value 67.960229
## iter  90 value 67.958250
## iter 100 value 67.923183
## final  value 67.923183 
## stopped after 100 iterations
## # weights:  49
## initial  value 198.249966 
## iter  10 value 85.874388
## iter  20 value 73.670774
## iter  30 value 67.339517
## iter  40 value 66.220155
## iter  50 value 65.099604
## iter  60 value 64.367001
## iter  70 value 61.793457
## iter  80 value 61.172629
## iter  90 value 55.499296
## iter 100 value 55.191648
## final  value 55.191648 
## stopped after 100 iterations
## # weights:  81
## initial  value 177.592743 
## iter  10 value 96.093389
## iter  20 value 80.390430
## iter  30 value 78.491160
## iter  40 value 78.390596
## iter  50 value 78.356179
## iter  60 value 78.104857
## iter  70 value 77.631184
## iter  80 value 77.573751
## iter  90 value 77.202495
## iter 100 value 76.481574
## final  value 76.481574 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.406665 
## iter  10 value 69.332427
## iter  20 value 60.915753
## iter  30 value 57.469910
## iter  40 value 56.607302
## iter  50 value 55.856639
## iter  60 value 54.564654
## iter  70 value 52.917403
## iter  80 value 52.827803
## iter  90 value 52.811941
## iter 100 value 52.808693
## final  value 52.808693 
## stopped after 100 iterations
## # weights:  49
## initial  value 207.100103 
## iter  10 value 110.000308
## final  value 110.000001 
## converged
## # weights:  81
## initial  value 166.313058 
## iter  10 value 93.092156
## iter  20 value 87.804768
## iter  30 value 75.882294
## iter  40 value 68.534637
## iter  50 value 67.585195
## iter  60 value 64.543462
## iter  70 value 61.751751
## iter  80 value 61.541596
## iter  90 value 61.541324
## iter 100 value 61.540950
## final  value 61.540950 
## stopped after 100 iterations
## # weights:  17
## initial  value 183.886545 
## iter  10 value 116.448522
## iter  20 value 76.940895
## iter  30 value 63.682097
## iter  40 value 63.266187
## iter  50 value 63.248820
## final  value 63.248576 
## converged
## # weights:  49
## initial  value 186.021304 
## iter  10 value 92.803887
## iter  20 value 66.382593
## iter  30 value 57.468089
## iter  40 value 55.027265
## iter  50 value 53.941152
## iter  60 value 53.113324
## iter  70 value 52.915554
## iter  80 value 52.892889
## iter  90 value 52.892670
## final  value 52.892665 
## converged
## # weights:  81
## initial  value 176.044319 
## iter  10 value 58.806861
## iter  20 value 47.141683
## iter  30 value 45.074083
## iter  40 value 44.648748
## iter  50 value 44.589615
## iter  60 value 44.557637
## iter  70 value 44.035750
## iter  80 value 43.919666
## iter  90 value 43.910585
## iter 100 value 43.909922
## final  value 43.909922 
## stopped after 100 iterations
## # weights:  17
## initial  value 182.655839 
## iter  10 value 67.090219
## iter  20 value 58.696907
## iter  30 value 57.286977
## iter  40 value 56.339902
## iter  50 value 55.864835
## iter  60 value 54.778676
## iter  70 value 54.308925
## iter  80 value 54.299293
## iter  90 value 54.286841
## iter 100 value 54.279432
## final  value 54.279432 
## stopped after 100 iterations
## # weights:  49
## initial  value 170.480915 
## iter  10 value 103.443559
## iter  20 value 72.710512
## iter  30 value 68.436293
## iter  40 value 59.875955
## iter  50 value 56.018838
## iter  60 value 55.397061
## iter  70 value 53.774633
## iter  80 value 52.618110
## iter  90 value 50.795917
## iter 100 value 49.689117
## final  value 49.689117 
## stopped after 100 iterations
## # weights:  81
## initial  value 189.510035 
## iter  10 value 103.613660
## iter  20 value 85.663633
## iter  30 value 82.411835
## iter  40 value 78.352086
## iter  50 value 77.864162
## iter  60 value 77.393499
## iter  70 value 76.187585
## iter  80 value 74.688208
## iter  90 value 74.439278
## iter 100 value 74.381268
## final  value 74.381268 
## stopped after 100 iterations
## # weights:  17
## initial  value 193.000674 
## iter  10 value 79.435809
## iter  20 value 70.335092
## iter  30 value 65.439878
## iter  40 value 62.587034
## iter  50 value 60.594915
## iter  60 value 59.921570
## iter  70 value 59.229973
## iter  80 value 58.444879
## iter  90 value 58.381579
## iter 100 value 58.343417
## final  value 58.343417 
## stopped after 100 iterations
## # weights:  49
## initial  value 180.188184 
## final  value 118.999991 
## converged
## # weights:  81
## initial  value 193.273834 
## iter  10 value 80.562966
## iter  20 value 73.973970
## iter  30 value 72.151362
## iter  40 value 71.990759
## iter  50 value 71.929476
## iter  60 value 57.703120
## iter  70 value 51.733706
## iter  80 value 48.224170
## iter  90 value 46.810690
## iter 100 value 45.932024
## final  value 45.932024 
## stopped after 100 iterations
## # weights:  17
## initial  value 173.397154 
## iter  10 value 84.650137
## iter  20 value 74.880565
## iter  30 value 70.579074
## iter  40 value 65.918888
## iter  50 value 65.176814
## final  value 65.172965 
## converged
## # weights:  49
## initial  value 171.512269 
## iter  10 value 82.150176
## iter  20 value 65.351352
## iter  30 value 59.004505
## iter  40 value 56.847031
## iter  50 value 50.272857
## iter  60 value 49.213996
## iter  70 value 49.079288
## iter  80 value 49.049600
## iter  90 value 49.011284
## iter 100 value 48.987749
## final  value 48.987749 
## stopped after 100 iterations
## # weights:  81
## initial  value 167.547822 
## iter  10 value 88.203595
## iter  20 value 57.034462
## iter  30 value 52.359706
## iter  40 value 51.263365
## iter  50 value 50.470069
## iter  60 value 49.556156
## iter  70 value 49.322606
## iter  80 value 49.294241
## iter  90 value 49.293073
## final  value 49.293056 
## converged
## # weights:  17
## initial  value 174.326071 
## iter  10 value 102.239630
## iter  20 value 88.973419
## iter  30 value 86.038829
## iter  40 value 80.732605
## iter  50 value 80.621963
## iter  60 value 79.711179
## iter  70 value 79.695319
## iter  80 value 78.774336
## iter  90 value 78.765835
## iter 100 value 78.754763
## final  value 78.754763 
## stopped after 100 iterations
## # weights:  49
## initial  value 182.505538 
## iter  10 value 98.522424
## iter  20 value 62.729165
## iter  30 value 45.072459
## iter  40 value 39.544253
## iter  50 value 38.894151
## iter  60 value 37.118056
## iter  70 value 35.949456
## iter  80 value 35.313520
## iter  90 value 35.192651
## iter 100 value 35.086222
## final  value 35.086222 
## stopped after 100 iterations
## # weights:  81
## initial  value 195.188294 
## iter  10 value 119.263583
## iter  20 value 119.254506
## iter  30 value 119.242464
## iter  40 value 64.302002
## iter  50 value 43.085984
## iter  60 value 34.475925
## iter  70 value 29.509648
## iter  80 value 28.508511
## iter  90 value 28.263357
## iter 100 value 28.096125
## final  value 28.096125 
## stopped after 100 iterations
## # weights:  17
## initial  value 169.728233 
## iter  10 value 86.009838
## iter  20 value 79.975684
## iter  30 value 78.218758
## iter  40 value 77.517459
## iter  50 value 76.410354
## iter  60 value 75.895475
## iter  70 value 75.268219
## iter  80 value 65.231157
## iter  90 value 64.954985
## iter 100 value 64.046042
## final  value 64.046042 
## stopped after 100 iterations
## # weights:  49
## initial  value 181.104480 
## iter  10 value 99.403564
## iter  20 value 74.358549
## iter  30 value 68.407088
## iter  40 value 64.034103
## iter  50 value 61.694719
## iter  60 value 60.868481
## iter  70 value 60.724978
## iter  80 value 60.240529
## iter  90 value 59.935910
## iter 100 value 59.639234
## final  value 59.639234 
## stopped after 100 iterations
## # weights:  81
## initial  value 191.804990 
## iter  10 value 91.812749
## iter  20 value 72.272917
## iter  30 value 61.336611
## iter  40 value 59.157607
## iter  50 value 57.788686
## iter  60 value 54.756200
## iter  70 value 54.081633
## iter  80 value 53.995334
## iter  90 value 53.773689
## iter 100 value 52.461788
## final  value 52.461788 
## stopped after 100 iterations
## # weights:  17
## initial  value 170.879014 
## iter  10 value 74.783998
## iter  20 value 68.035462
## iter  30 value 67.818860
## final  value 67.818219 
## converged
## # weights:  49
## initial  value 189.589513 
## iter  10 value 99.686717
## iter  20 value 67.597771
## iter  30 value 64.506732
## iter  40 value 60.595165
## iter  50 value 58.194500
## iter  60 value 57.656386
## iter  70 value 57.220370
## iter  80 value 57.043384
## iter  90 value 56.936031
## iter 100 value 56.929679
## final  value 56.929679 
## stopped after 100 iterations
## # weights:  81
## initial  value 184.700114 
## iter  10 value 73.289358
## iter  20 value 56.364874
## iter  30 value 53.434787
## iter  40 value 47.291422
## iter  50 value 45.790732
## iter  60 value 45.756744
## iter  70 value 45.756128
## final  value 45.756119 
## converged
## # weights:  17
## initial  value 178.881175 
## iter  10 value 119.357687
## iter  20 value 117.369588
## iter  30 value 117.291944
## iter  40 value 112.059800
## iter  50 value 100.898052
## iter  60 value 88.184396
## iter  70 value 81.601703
## iter  80 value 80.932926
## iter  90 value 79.389991
## iter 100 value 79.273184
## final  value 79.273184 
## stopped after 100 iterations
## # weights:  49
## initial  value 180.033235 
## iter  10 value 110.574337
## iter  20 value 107.755594
## iter  30 value 107.266444
## iter  40 value 105.273931
## iter  50 value 103.396137
## iter  60 value 101.270390
## iter  70 value 99.251086
## iter  80 value 96.861527
## iter  90 value 96.169846
## iter 100 value 95.177828
## final  value 95.177828 
## stopped after 100 iterations
## # weights:  81
## initial  value 158.183317 
## iter  10 value 93.121746
## iter  20 value 86.876263
## iter  30 value 86.082652
## iter  40 value 84.278203
## iter  50 value 83.301829
## iter  60 value 83.237518
## iter  70 value 82.321297
## iter  80 value 82.248107
## iter  90 value 82.224868
## iter 100 value 82.189874
## final  value 82.189874 
## stopped after 100 iterations
## # weights:  17
## initial  value 179.008256 
## iter  10 value 75.145038
## iter  20 value 71.107564
## iter  30 value 69.171181
## iter  40 value 69.167016
## iter  50 value 64.361829
## iter  60 value 62.441224
## iter  70 value 61.733698
## iter  80 value 60.783172
## iter  90 value 59.903069
## iter 100 value 59.829612
## final  value 59.829612 
## stopped after 100 iterations
## # weights:  49
## initial  value 177.747140 
## iter  10 value 96.493705
## iter  20 value 74.692992
## iter  30 value 73.961907
## iter  40 value 72.560568
## iter  50 value 70.729418
## iter  60 value 70.559615
## iter  70 value 70.380319
## iter  80 value 70.299105
## iter  90 value 70.254817
## iter 100 value 70.118241
## final  value 70.118241 
## stopped after 100 iterations
## # weights:  81
## initial  value 174.805562 
## iter  10 value 96.951502
## iter  20 value 83.873344
## iter  30 value 79.206497
## iter  40 value 77.372279
## iter  50 value 76.376320
## iter  60 value 74.576152
## iter  70 value 74.309572
## iter  80 value 73.306192
## iter  90 value 73.295084
## iter 100 value 72.743681
## final  value 72.743681 
## stopped after 100 iterations
## # weights:  17
## initial  value 173.557817 
## iter  10 value 72.574253
## iter  20 value 59.858040
## iter  30 value 58.956791
## iter  40 value 58.890242
## iter  50 value 58.890030
## final  value 58.890029 
## converged
## # weights:  49
## initial  value 179.250690 
## iter  10 value 72.824029
## iter  20 value 59.088036
## iter  30 value 55.635658
## iter  40 value 54.109578
## iter  50 value 53.669009
## iter  60 value 53.054029
## iter  70 value 52.896477
## iter  80 value 52.893967
## final  value 52.893959 
## converged
## # weights:  81
## initial  value 175.476038 
## iter  10 value 76.686283
## iter  20 value 52.764162
## iter  30 value 48.002387
## iter  40 value 44.813480
## iter  50 value 44.011716
## iter  60 value 42.620878
## iter  70 value 42.329287
## iter  80 value 42.282225
## iter  90 value 42.245163
## iter 100 value 42.238796
## final  value 42.238796 
## stopped after 100 iterations
## # weights:  17
## initial  value 189.896341 
## iter  10 value 73.732245
## iter  20 value 64.552092
## iter  30 value 64.265598
## iter  40 value 64.142333
## iter  50 value 64.134848
## iter  60 value 64.132764
## iter  70 value 64.121438
## iter  80 value 64.113167
## iter  90 value 63.933340
## iter 100 value 58.256253
## final  value 58.256253 
## stopped after 100 iterations
## # weights:  49
## initial  value 186.260537 
## iter  10 value 79.683086
## iter  20 value 72.233239
## iter  30 value 71.246736
## iter  40 value 69.675633
## iter  50 value 69.254512
## iter  60 value 69.237994
## iter  70 value 69.221109
## iter  80 value 68.239490
## iter  90 value 68.187535
## iter 100 value 68.180113
## final  value 68.180113 
## stopped after 100 iterations
## # weights:  81
## initial  value 146.016393 
## iter  10 value 88.341479
## iter  20 value 80.298978
## iter  30 value 77.277672
## iter  40 value 72.306098
## iter  50 value 72.214511
## iter  60 value 69.793017
## iter  70 value 69.270588
## iter  80 value 69.240468
## iter  90 value 69.213669
## iter 100 value 69.184276
## final  value 69.184276 
## stopped after 100 iterations
## # weights:  17
## initial  value 182.742960 
## iter  10 value 101.606581
## iter  20 value 94.061636
## iter  30 value 92.278167
## iter  40 value 92.247949
## iter  50 value 92.102505
## iter  60 value 90.230296
## iter  70 value 88.167150
## iter  80 value 86.294664
## iter  90 value 84.702691
## iter 100 value 84.690033
## final  value 84.690033 
## stopped after 100 iterations
## # weights:  49
## initial  value 190.338090 
## iter  10 value 114.998708
## final  value 113.999984 
## converged
## # weights:  81
## initial  value 161.487830 
## iter  10 value 84.502375
## iter  20 value 77.565854
## iter  30 value 74.618212
## iter  40 value 74.011470
## iter  50 value 72.963224
## iter  60 value 69.017825
## iter  70 value 65.437553
## iter  80 value 64.064385
## iter  90 value 63.034962
## iter 100 value 63.003445
## final  value 63.003445 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.584947 
## iter  10 value 96.694753
## iter  20 value 77.288492
## iter  30 value 70.202176
## iter  40 value 65.158096
## iter  50 value 65.109349
## final  value 65.108127 
## converged
## # weights:  49
## initial  value 183.357667 
## iter  10 value 68.946090
## iter  20 value 59.241134
## iter  30 value 56.332280
## iter  40 value 55.598127
## iter  50 value 53.751095
## iter  60 value 53.383251
## iter  70 value 52.850634
## iter  80 value 52.718420
## iter  90 value 52.710193
## final  value 52.710130 
## converged
## # weights:  81
## initial  value 193.041333 
## iter  10 value 88.963957
## iter  20 value 71.224302
## iter  30 value 61.649402
## iter  40 value 56.103278
## iter  50 value 53.634205
## iter  60 value 52.488870
## iter  70 value 52.056813
## iter  80 value 51.762911
## iter  90 value 51.121100
## iter 100 value 50.037840
## final  value 50.037840 
## stopped after 100 iterations
## # weights:  17
## initial  value 172.562078 
## iter  10 value 71.338990
## iter  20 value 61.064155
## iter  30 value 58.971035
## iter  40 value 56.428167
## iter  50 value 55.764172
## iter  60 value 55.601504
## iter  70 value 54.506582
## iter  80 value 54.498587
## iter  90 value 54.497334
## iter 100 value 54.497065
## final  value 54.497065 
## stopped after 100 iterations
## # weights:  49
## initial  value 199.793180 
## iter  10 value 81.397481
## iter  20 value 78.278831
## iter  30 value 77.276121
## iter  40 value 76.827027
## iter  50 value 74.698928
## iter  60 value 74.085359
## iter  70 value 71.740428
## iter  80 value 70.150746
## iter  90 value 68.221143
## iter 100 value 68.103238
## final  value 68.103238 
## stopped after 100 iterations
## # weights:  81
## initial  value 175.160295 
## iter  10 value 96.897195
## iter  20 value 63.524076
## iter  30 value 55.823076
## iter  40 value 53.175002
## iter  50 value 51.627680
## iter  60 value 51.436148
## iter  70 value 43.882837
## iter  80 value 40.610053
## iter  90 value 39.891654
## iter 100 value 39.062351
## final  value 39.062351 
## stopped after 100 iterations
## # weights:  17
## initial  value 178.679255 
## iter  10 value 111.175022
## iter  20 value 93.987485
## iter  30 value 91.936029
## iter  40 value 85.879972
## iter  50 value 84.704546
## iter  60 value 84.540269
## iter  70 value 77.704275
## iter  80 value 77.692560
## iter  90 value 77.690398
## iter 100 value 77.219958
## final  value 77.219958 
## stopped after 100 iterations
## # weights:  49
## initial  value 198.776960 
## iter  10 value 91.612358
## iter  20 value 87.835579
## iter  30 value 79.118889
## iter  40 value 73.084813
## iter  50 value 68.095850
## iter  60 value 67.899062
## iter  70 value 64.981283
## iter  80 value 64.852802
## iter  90 value 64.004463
## iter 100 value 63.111482
## final  value 63.111482 
## stopped after 100 iterations
## # weights:  81
## initial  value 201.735799 
## iter  10 value 102.700286
## iter  20 value 75.076499
## iter  30 value 71.995754
## iter  40 value 70.986710
## iter  50 value 70.942599
## iter  60 value 70.925799
## iter  70 value 70.875847
## iter  80 value 69.808442
## iter  90 value 69.616384
## iter 100 value 67.084379
## final  value 67.084379 
## stopped after 100 iterations
## # weights:  17
## initial  value 172.257489 
## iter  10 value 80.748671
## iter  20 value 64.501938
## iter  30 value 63.202886
## final  value 63.185001 
## converged
## # weights:  49
## initial  value 181.075049 
## iter  10 value 67.618673
## iter  20 value 55.879420
## iter  30 value 51.835260
## iter  40 value 51.091256
## iter  50 value 51.059529
## iter  60 value 51.045511
## final  value 51.045506 
## converged
## # weights:  81
## initial  value 195.895217 
## iter  10 value 109.763995
## iter  20 value 71.294547
## iter  30 value 52.425371
## iter  40 value 47.918530
## iter  50 value 45.216660
## iter  60 value 43.788015
## iter  70 value 43.162790
## iter  80 value 43.057980
## iter  90 value 43.049150
## iter 100 value 43.048925
## final  value 43.048925 
## stopped after 100 iterations
## # weights:  17
## initial  value 179.124037 
## iter  10 value 68.782691
## iter  20 value 56.108919
## iter  30 value 55.044800
## iter  40 value 53.196704
## iter  50 value 52.343043
## iter  60 value 51.435535
## iter  70 value 50.923904
## iter  80 value 50.548376
## iter  90 value 50.019312
## iter 100 value 49.970503
## final  value 49.970503 
## stopped after 100 iterations
## # weights:  49
## initial  value 171.874325 
## iter  10 value 88.822652
## iter  20 value 44.022332
## iter  30 value 25.856595
## iter  40 value 24.218729
## iter  50 value 24.161456
## iter  60 value 24.064750
## iter  70 value 23.878895
## iter  80 value 23.646760
## iter  90 value 23.491605
## iter 100 value 22.465641
## final  value 22.465641 
## stopped after 100 iterations
## # weights:  81
## initial  value 164.568500 
## iter  10 value 107.183239
## iter  20 value 102.138622
## iter  30 value 96.997692
## iter  40 value 91.037434
## iter  50 value 88.530968
## iter  60 value 86.905480
## iter  70 value 86.838623
## iter  80 value 86.350717
## iter  90 value 84.998841
## iter 100 value 84.205531
## final  value 84.205531 
## stopped after 100 iterations
## # weights:  17
## initial  value 172.072183 
## iter  10 value 93.785422
## iter  20 value 84.175495
## iter  30 value 82.951297
## iter  40 value 82.634163
## iter  50 value 82.618587
## iter  60 value 82.556031
## iter  70 value 82.542656
## iter  80 value 82.534807
## iter  90 value 82.532051
## iter 100 value 82.530772
## final  value 82.530772 
## stopped after 100 iterations
## # weights:  49
## initial  value 168.387829 
## iter  10 value 101.747170
## iter  20 value 86.112218
## iter  30 value 80.412067
## iter  40 value 79.187570
## iter  50 value 77.895893
## iter  60 value 76.363810
## iter  70 value 75.197850
## iter  80 value 73.434030
## iter  90 value 72.106411
## iter 100 value 72.078154
## final  value 72.078154 
## stopped after 100 iterations
## # weights:  81
## initial  value 181.487173 
## iter  10 value 84.971164
## iter  20 value 60.589932
## iter  30 value 56.101634
## iter  40 value 51.596441
## iter  50 value 47.403698
## iter  60 value 45.798036
## iter  70 value 45.189777
## iter  80 value 40.007925
## iter  90 value 36.685230
## iter 100 value 35.102977
## final  value 35.102977 
## stopped after 100 iterations
## # weights:  17
## initial  value 178.558423 
## iter  10 value 71.737412
## iter  20 value 64.972882
## iter  30 value 64.015764
## final  value 64.006591 
## converged
## # weights:  49
## initial  value 168.515120 
## iter  10 value 75.496919
## iter  20 value 57.683576
## iter  30 value 55.030934
## iter  40 value 51.293644
## iter  50 value 49.649891
## iter  60 value 48.153490
## iter  70 value 47.609398
## iter  80 value 47.597370
## final  value 47.597368 
## converged
## # weights:  81
## initial  value 187.585354 
## iter  10 value 67.713914
## iter  20 value 55.757477
## iter  30 value 53.552722
## iter  40 value 53.301612
## iter  50 value 52.986262
## iter  60 value 50.601659
## iter  70 value 50.253008
## iter  80 value 49.841068
## iter  90 value 49.571610
## iter 100 value 49.484627
## final  value 49.484627 
## stopped after 100 iterations
## # weights:  17
## initial  value 174.879336 
## iter  10 value 91.657866
## iter  20 value 77.690750
## iter  30 value 71.467563
## iter  40 value 68.297927
## iter  50 value 68.034626
## iter  60 value 67.911185
## iter  70 value 67.907149
## iter  80 value 67.903774
## iter  90 value 67.902186
## iter 100 value 67.901711
## final  value 67.901711 
## stopped after 100 iterations
## # weights:  49
## initial  value 168.672871 
## iter  10 value 116.258446
## iter  20 value 105.524149
## iter  30 value 93.105167
## iter  40 value 86.371251
## iter  50 value 84.526567
## iter  60 value 79.876188
## iter  70 value 76.756318
## iter  80 value 73.385223
## iter  90 value 72.570546
## iter 100 value 69.408009
## final  value 69.408009 
## stopped after 100 iterations
## # weights:  81
## initial  value 160.065272 
## iter  10 value 67.354120
## iter  20 value 38.608555
## iter  30 value 31.791862
## iter  40 value 30.564113
## iter  50 value 30.100069
## iter  60 value 28.971673
## iter  70 value 28.313625
## iter  80 value 27.061041
## iter  90 value 26.287177
## iter 100 value 25.712418
## final  value 25.712418 
## stopped after 100 iterations
## # weights:  17
## initial  value 172.592134 
## iter  10 value 99.625751
## iter  20 value 98.633305
## iter  30 value 97.665392
## iter  40 value 95.444969
## iter  50 value 91.840213
## iter  60 value 90.753384
## iter  70 value 90.729435
## iter  80 value 89.860197
## iter  90 value 89.713280
## iter 100 value 89.710571
## final  value 89.710571 
## stopped after 100 iterations
## # weights:  49
## initial  value 191.620159 
## iter  10 value 103.592280
## iter  20 value 85.698456
## iter  30 value 84.116943
## iter  40 value 82.756198
## iter  50 value 72.588388
## iter  60 value 67.954580
## iter  70 value 61.763608
## iter  80 value 59.741993
## iter  90 value 58.446661
## iter 100 value 57.409681
## final  value 57.409681 
## stopped after 100 iterations
## # weights:  81
## initial  value 166.131884 
## final  value 112.999999 
## converged
## # weights:  17
## initial  value 179.919662 
## iter  10 value 82.808248
## iter  20 value 74.194069
## iter  30 value 68.258848
## iter  40 value 63.583666
## final  value 63.576867 
## converged
## # weights:  49
## initial  value 170.120509 
## iter  10 value 66.487928
## iter  20 value 58.049269
## iter  30 value 52.501607
## iter  40 value 49.895196
## iter  50 value 49.265845
## iter  60 value 49.237698
## iter  70 value 49.237490
## final  value 49.237488 
## converged
## # weights:  81
## initial  value 196.742864 
## iter  10 value 94.622323
## iter  20 value 60.298834
## iter  30 value 51.512970
## iter  40 value 48.036352
## iter  50 value 46.955691
## iter  60 value 46.240349
## iter  70 value 45.950190
## iter  80 value 45.380090
## iter  90 value 44.247860
## iter 100 value 44.131664
## final  value 44.131664 
## stopped after 100 iterations
## # weights:  17
## initial  value 199.167654 
## iter  10 value 105.470724
## iter  20 value 93.983407
## iter  30 value 84.065713
## iter  40 value 76.536127
## iter  50 value 66.195557
## iter  60 value 65.687334
## iter  70 value 65.669748
## iter  80 value 65.666542
## iter  90 value 65.662293
## iter 100 value 65.657946
## final  value 65.657946 
## stopped after 100 iterations
## # weights:  49
## initial  value 169.882513 
## iter  10 value 109.363084
## iter  20 value 96.404907
## iter  30 value 91.234828
## iter  40 value 91.036715
## iter  50 value 90.024062
## iter  60 value 89.900710
## iter  70 value 88.669081
## iter  80 value 84.489782
## iter  90 value 84.388176
## iter 100 value 84.346000
## final  value 84.346000 
## stopped after 100 iterations
## # weights:  81
## initial  value 189.640529 
## iter  10 value 109.884303
## iter  20 value 92.484063
## iter  30 value 80.805631
## iter  40 value 77.313095
## iter  50 value 75.271144
## iter  60 value 72.092678
## iter  70 value 70.796795
## iter  80 value 67.985722
## iter  90 value 66.052408
## iter 100 value 63.169709
## final  value 63.169709 
## stopped after 100 iterations
## # weights:  17
## initial  value 192.306952 
## iter  10 value 95.662147
## iter  20 value 73.479583
## iter  30 value 70.913966
## iter  40 value 68.962296
## iter  50 value 67.695450
## iter  60 value 66.691113
## iter  70 value 66.597747
## iter  80 value 65.684082
## iter  90 value 65.549371
## iter 100 value 65.524198
## final  value 65.524198 
## stopped after 100 iterations
## # weights:  49
## initial  value 219.944326 
## iter  10 value 105.894979
## iter  20 value 91.991295
## iter  30 value 86.972693
## iter  40 value 85.276252
## iter  50 value 81.894356
## iter  60 value 80.850366
## iter  70 value 79.830974
## iter  80 value 79.815568
## iter  90 value 79.426410
## iter 100 value 77.925223
## final  value 77.925223 
## stopped after 100 iterations
## # weights:  81
## initial  value 185.101553 
## iter  10 value 78.294745
## iter  20 value 70.074779
## iter  30 value 62.184226
## iter  40 value 57.500041
## iter  50 value 52.708140
## iter  60 value 47.885403
## iter  70 value 47.436178
## iter  80 value 47.256098
## iter  90 value 46.850345
## iter 100 value 46.048368
## final  value 46.048368 
## stopped after 100 iterations
## # weights:  17
## initial  value 169.491794 
## iter  10 value 70.848923
## iter  20 value 63.266680
## iter  30 value 63.154283
## final  value 63.154229 
## converged
## # weights:  49
## initial  value 210.671569 
## iter  10 value 79.373787
## iter  20 value 63.549847
## iter  30 value 58.424389
## iter  40 value 56.944442
## iter  50 value 53.532693
## iter  60 value 51.871125
## iter  70 value 51.478329
## iter  80 value 51.468547
## final  value 51.468459 
## converged
## # weights:  81
## initial  value 211.859671 
## iter  10 value 63.285910
## iter  20 value 49.702331
## iter  30 value 46.526386
## iter  40 value 45.572893
## iter  50 value 45.126671
## iter  60 value 44.792015
## iter  70 value 44.714420
## iter  80 value 44.671689
## iter  90 value 44.670829
## final  value 44.670826 
## converged
## # weights:  17
## initial  value 176.951392 
## iter  10 value 84.226294
## iter  20 value 64.607696
## iter  30 value 54.929183
## iter  40 value 53.559752
## iter  50 value 53.225195
## iter  60 value 53.093383
## iter  70 value 52.956676
## final  value 52.940522 
## converged
## # weights:  49
## initial  value 186.222718 
## iter  10 value 97.374004
## iter  20 value 97.270023
## iter  30 value 88.679398
## iter  40 value 85.711096
## iter  50 value 82.271904
## iter  60 value 81.728928
## iter  70 value 81.579817
## iter  80 value 79.432652
## iter  90 value 77.919717
## iter 100 value 74.470643
## final  value 74.470643 
## stopped after 100 iterations
## # weights:  81
## initial  value 189.011444 
## iter  10 value 106.589751
## iter  20 value 99.343371
## iter  30 value 92.197175
## iter  40 value 85.013658
## iter  50 value 75.715067
## iter  60 value 74.175065
## iter  70 value 74.045708
## iter  80 value 73.171110
## iter  90 value 73.159656
## iter 100 value 73.140923
## final  value 73.140923 
## stopped after 100 iterations
## # weights:  17
## initial  value 207.301208 
## iter  10 value 66.373670
## iter  20 value 60.286169
## iter  30 value 59.300641
## iter  40 value 58.351574
## iter  50 value 58.190796
## iter  60 value 57.568385
## iter  70 value 57.517155
## iter  80 value 57.514182
## iter  90 value 57.513977
## iter 100 value 57.513924
## final  value 57.513924 
## stopped after 100 iterations
## # weights:  49
## initial  value 183.590527 
## iter  10 value 109.301824
## iter  20 value 106.634144
## iter  30 value 102.514844
## iter  40 value 84.170884
## iter  50 value 75.142303
## iter  60 value 67.137646
## iter  70 value 61.654159
## iter  80 value 61.371866
## iter  90 value 52.998159
## iter 100 value 51.000742
## final  value 51.000742 
## stopped after 100 iterations
## # weights:  81
## initial  value 181.488296 
## final  value 117.000000 
## converged
## # weights:  17
## initial  value 191.348153 
## iter  10 value 112.691535
## iter  20 value 75.581675
## iter  30 value 64.202085
## iter  40 value 62.824348
## final  value 62.794951 
## converged
## # weights:  49
## initial  value 175.205034 
## iter  10 value 69.942884
## iter  20 value 61.105847
## iter  30 value 58.565369
## iter  40 value 54.087853
## iter  50 value 53.866728
## iter  60 value 52.939720
## iter  70 value 51.002677
## iter  80 value 49.181145
## iter  90 value 48.834667
## iter 100 value 48.765235
## final  value 48.765235 
## stopped after 100 iterations
## # weights:  81
## initial  value 185.075591 
## iter  10 value 74.790249
## iter  20 value 55.853763
## iter  30 value 51.030267
## iter  40 value 48.633315
## iter  50 value 46.895278
## iter  60 value 46.147770
## iter  70 value 45.730047
## iter  80 value 45.521307
## iter  90 value 44.877851
## iter 100 value 44.746787
## final  value 44.746787 
## stopped after 100 iterations
## # weights:  17
## initial  value 183.320334 
## iter  10 value 88.127110
## iter  20 value 87.110717
## iter  30 value 87.107363
## iter  40 value 87.102913
## iter  50 value 87.096536
## iter  60 value 87.086281
## iter  70 value 87.066224
## iter  80 value 87.006305
## iter  90 value 85.136814
## iter 100 value 72.239246
## final  value 72.239246 
## stopped after 100 iterations
## # weights:  49
## initial  value 196.112358 
## iter  10 value 75.680236
## iter  20 value 65.048062
## iter  30 value 60.759922
## iter  40 value 53.071203
## iter  50 value 43.216300
## iter  60 value 37.680885
## iter  70 value 35.877451
## iter  80 value 35.433554
## iter  90 value 35.078061
## iter 100 value 34.945747
## final  value 34.945747 
## stopped after 100 iterations
## # weights:  81
## initial  value 154.354739 
## iter  10 value 84.469644
## iter  20 value 74.734589
## iter  30 value 66.587104
## iter  40 value 56.391673
## iter  50 value 51.401367
## iter  60 value 49.551876
## iter  70 value 46.096485
## iter  80 value 42.631508
## iter  90 value 41.225288
## iter 100 value 40.427053
## final  value 40.427053 
## stopped after 100 iterations
## # weights:  17
## initial  value 173.645833 
## iter  10 value 75.880016
## iter  20 value 71.160696
## iter  30 value 70.247485
## iter  40 value 69.985340
## iter  50 value 69.973577
## iter  60 value 69.904683
## iter  70 value 69.764882
## iter  80 value 69.730695
## iter  90 value 69.726708
## iter 100 value 69.725156
## final  value 69.725156 
## stopped after 100 iterations
## # weights:  49
## initial  value 195.635840 
## iter  10 value 95.714243
## iter  20 value 73.863749
## iter  30 value 72.319830
## iter  40 value 69.900850
## iter  50 value 66.099637
## iter  60 value 65.926263
## iter  70 value 65.881536
## iter  80 value 64.884042
## iter  90 value 55.983275
## iter 100 value 55.934041
## final  value 55.934041 
## stopped after 100 iterations
## # weights:  81
## initial  value 178.751324 
## iter  10 value 71.779264
## iter  20 value 66.022553
## iter  30 value 65.980936
## iter  40 value 64.166334
## iter  50 value 62.077803
## iter  60 value 62.004161
## iter  70 value 61.002015
## iter  80 value 60.002970
## iter  90 value 59.779165
## iter 100 value 58.893504
## final  value 58.893504 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.211449 
## iter  10 value 84.934714
## iter  20 value 67.959222
## iter  30 value 63.890975
## iter  40 value 61.414823
## iter  50 value 61.156021
## final  value 61.155087 
## converged
## # weights:  49
## initial  value 188.951838 
## iter  10 value 69.305616
## iter  20 value 59.621113
## iter  30 value 55.688978
## iter  40 value 54.976779
## iter  50 value 54.898093
## iter  60 value 54.897336
## final  value 54.897297 
## converged
## # weights:  81
## initial  value 160.119448 
## iter  10 value 70.323362
## iter  20 value 53.392514
## iter  30 value 50.749473
## iter  40 value 48.853307
## iter  50 value 48.130574
## iter  60 value 47.986605
## iter  70 value 47.972247
## iter  80 value 47.963495
## iter  90 value 47.699673
## iter 100 value 47.545108
## final  value 47.545108 
## stopped after 100 iterations
## # weights:  17
## initial  value 167.463170 
## iter  10 value 89.084058
## iter  20 value 87.197376
## iter  30 value 84.071166
## iter  40 value 82.508118
## iter  50 value 78.638002
## iter  60 value 72.098018
## iter  70 value 71.879394
## iter  80 value 71.850430
## iter  90 value 71.840884
## iter 100 value 71.821381
## final  value 71.821381 
## stopped after 100 iterations
## # weights:  49
## initial  value 223.025936 
## iter  10 value 88.515862
## iter  20 value 86.183194
## iter  30 value 84.659140
## iter  40 value 81.360965
## iter  50 value 75.849013
## iter  60 value 72.424388
## iter  70 value 69.980710
## iter  80 value 69.325698
## iter  90 value 68.455215
## iter 100 value 67.490698
## final  value 67.490698 
## stopped after 100 iterations
## # weights:  81
## initial  value 179.075968 
## iter  10 value 82.490218
## iter  20 value 71.404311
## iter  30 value 70.355129
## iter  40 value 69.355859
## iter  50 value 69.326103
## iter  60 value 69.320738
## iter  70 value 68.317650
## iter  80 value 68.253359
## iter  90 value 67.919627
## iter 100 value 66.731332
## final  value 66.731332 
## stopped after 100 iterations
## # weights:  17
## initial  value 187.398943 
## iter  10 value 99.067816
## iter  20 value 83.412649
## iter  30 value 80.133682
## iter  40 value 79.996140
## iter  50 value 79.990368
## iter  60 value 78.981743
## iter  70 value 77.975300
## iter  80 value 77.009011
## iter  90 value 76.814149
## iter 100 value 76.706255
## final  value 76.706255 
## stopped after 100 iterations
## # weights:  49
## initial  value 191.203213 
## iter  10 value 115.993271
## iter  20 value 115.992234
## iter  30 value 115.990821
## iter  40 value 115.988784
## iter  50 value 115.985595
## iter  60 value 115.979906
## iter  70 value 115.966977
## iter  80 value 115.911411
## iter  90 value 101.476353
## iter 100 value 100.250916
## final  value 100.250916 
## stopped after 100 iterations
## # weights:  81
## initial  value 176.942806 
## iter  10 value 97.843817
## iter  20 value 92.321687
## iter  30 value 88.914718
## iter  40 value 87.984690
## iter  50 value 86.658735
## iter  60 value 85.584971
## iter  70 value 85.277321
## iter  80 value 84.291696
## iter  90 value 82.839432
## iter 100 value 81.567306
## final  value 81.567306 
## stopped after 100 iterations
## # weights:  17
## initial  value 186.471259 
## iter  10 value 85.024825
## iter  20 value 72.634503
## iter  30 value 63.997527
## iter  40 value 63.581680
## iter  50 value 63.579336
## final  value 63.579328 
## converged
## # weights:  49
## initial  value 169.500713 
## iter  10 value 76.629919
## iter  20 value 64.358245
## iter  30 value 60.985424
## iter  40 value 56.381087
## iter  50 value 55.320317
## iter  60 value 54.395847
## iter  70 value 52.228950
## iter  80 value 50.845870
## iter  90 value 49.867503
## iter 100 value 49.842581
## final  value 49.842581 
## stopped after 100 iterations
## # weights:  81
## initial  value 202.476336 
## iter  10 value 72.010272
## iter  20 value 58.601571
## iter  30 value 54.710997
## iter  40 value 53.047054
## iter  50 value 51.549027
## iter  60 value 51.170374
## iter  70 value 49.870931
## iter  80 value 49.132848
## iter  90 value 48.951812
## iter 100 value 48.904696
## final  value 48.904696 
## stopped after 100 iterations
## # weights:  17
## initial  value 175.451052 
## iter  10 value 83.058261
## iter  20 value 81.418166
## iter  30 value 80.054118
## iter  40 value 78.102761
## iter  50 value 77.196490
## iter  60 value 77.069022
## iter  70 value 77.058996
## iter  80 value 77.051364
## iter  90 value 77.050087
## iter 100 value 77.047391
## final  value 77.047391 
## stopped after 100 iterations
## # weights:  49
## initial  value 184.066919 
## iter  10 value 88.609108
## iter  20 value 83.202954
## iter  30 value 79.229640
## iter  40 value 78.223622
## iter  50 value 77.209937
## iter  60 value 75.098498
## iter  70 value 68.838952
## iter  80 value 62.129535
## iter  90 value 56.631658
## iter 100 value 54.005960
## final  value 54.005960 
## stopped after 100 iterations
## # weights:  81
## initial  value 172.514941 
## iter  10 value 93.356242
## iter  20 value 80.641716
## iter  30 value 77.461104
## iter  40 value 76.417345
## iter  50 value 75.367647
## iter  60 value 74.417278
## iter  70 value 73.410274
## iter  80 value 72.376233
## iter  90 value 72.251456
## iter 100 value 71.133241
## final  value 71.133241 
## stopped after 100 iterations
## # weights:  17
## initial  value 175.657623 
## iter  10 value 71.604382
## iter  20 value 59.105671
## iter  30 value 57.380387
## iter  40 value 56.414040
## iter  50 value 55.567821
## iter  60 value 52.433654
## iter  70 value 50.693611
## iter  80 value 50.674105
## iter  90 value 50.673826
## final  value 50.673821 
## converged
## # weights:  49
## initial  value 186.852635 
## iter  10 value 115.999623
## iter  20 value 115.999609
## iter  30 value 115.999591
## iter  40 value 115.999568
## iter  50 value 115.999534
## iter  60 value 115.999481
## iter  70 value 115.999376
## iter  80 value 115.999017
## iter  90 value 114.999734
## final  value 114.999573 
## converged
## # weights:  81
## initial  value 195.378609 
## iter  10 value 98.565078
## iter  20 value 88.950292
## iter  30 value 87.998660
## iter  40 value 87.997793
## iter  50 value 87.997275
## iter  60 value 87.995624
## iter  70 value 86.997617
## iter  80 value 86.986137
## iter  90 value 85.659677
## iter 100 value 78.513480
## final  value 78.513480 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.558622 
## iter  10 value 81.670883
## iter  20 value 74.276748
## iter  30 value 68.554343
## iter  40 value 65.677217
## iter  50 value 65.173509
## final  value 65.166574 
## converged
## # weights:  49
## initial  value 193.465361 
## iter  10 value 119.458120
## iter  20 value 66.267546
## iter  30 value 58.112427
## iter  40 value 51.525289
## iter  50 value 50.237213
## iter  60 value 49.769317
## iter  70 value 49.652288
## iter  80 value 49.648714
## iter  90 value 49.648674
## iter  90 value 49.648673
## iter  90 value 49.648673
## final  value 49.648673 
## converged
## # weights:  81
## initial  value 173.377104 
## iter  10 value 77.710943
## iter  20 value 58.676711
## iter  30 value 53.301086
## iter  40 value 50.547685
## iter  50 value 49.440565
## iter  60 value 48.515717
## iter  70 value 46.633395
## iter  80 value 46.351258
## iter  90 value 46.340869
## iter 100 value 46.340679
## final  value 46.340679 
## stopped after 100 iterations
## # weights:  17
## initial  value 183.074463 
## iter  10 value 104.071559
## iter  20 value 102.000792
## iter  30 value 100.783668
## iter  40 value 87.224292
## iter  50 value 84.589883
## iter  60 value 83.254915
## iter  70 value 81.888408
## iter  80 value 81.776317
## iter  90 value 81.752270
## iter 100 value 80.849127
## final  value 80.849127 
## stopped after 100 iterations
## # weights:  49
## initial  value 178.396991 
## iter  10 value 106.969217
## iter  20 value 95.500335
## iter  30 value 94.436916
## iter  40 value 94.427420
## iter  50 value 94.174518
## iter  60 value 93.181700
## iter  70 value 92.236152
## iter  80 value 91.156394
## iter  90 value 91.046723
## iter 100 value 89.940695
## final  value 89.940695 
## stopped after 100 iterations
## # weights:  81
## initial  value 201.382590 
## iter  10 value 104.022301
## iter  20 value 98.324420
## iter  30 value 98.281558
## iter  40 value 95.433546
## iter  50 value 92.204134
## iter  60 value 90.280949
## iter  70 value 89.197577
## iter  80 value 89.075320
## iter  90 value 84.615270
## iter 100 value 83.565797
## final  value 83.565797 
## stopped after 100 iterations
## # weights:  17
## initial  value 190.336242 
## iter  10 value 90.009266
## iter  20 value 88.277813
## iter  30 value 87.115435
## iter  40 value 85.047441
## iter  50 value 84.290766
## iter  60 value 84.013241
## iter  70 value 84.003836
## iter  80 value 83.010307
## iter  90 value 83.008123
## iter 100 value 83.006044
## final  value 83.006044 
## stopped after 100 iterations
## # weights:  49
## initial  value 160.401269 
## iter  10 value 112.000505
## iter  20 value 111.999109
## iter  30 value 108.992549
## iter  40 value 108.800107
## iter  50 value 108.799702
## iter  60 value 107.063230
## iter  70 value 102.973400
## iter  80 value 101.332294
## iter  90 value 101.178998
## iter 100 value 101.161687
## final  value 101.161687 
## stopped after 100 iterations
## # weights:  81
## initial  value 194.126083 
## iter  10 value 80.679446
## iter  20 value 75.315304
## iter  30 value 71.279511
## iter  40 value 70.529731
## iter  50 value 69.705700
## iter  60 value 68.998069
## iter  70 value 66.999944
## iter  80 value 66.965221
## iter  90 value 66.739777
## iter 100 value 58.714513
## final  value 58.714513 
## stopped after 100 iterations
## # weights:  17
## initial  value 175.589118 
## iter  10 value 81.667366
## iter  20 value 75.618821
## iter  30 value 66.207909
## iter  40 value 65.951785
## iter  50 value 65.939658
## final  value 65.939553 
## converged
## # weights:  49
## initial  value 173.138881 
## iter  10 value 83.018436
## iter  20 value 63.157009
## iter  30 value 56.911930
## iter  40 value 54.662109
## iter  50 value 53.912913
## iter  60 value 53.602118
## iter  70 value 53.577633
## iter  80 value 53.575408
## final  value 53.575400 
## converged
## # weights:  81
## initial  value 162.427178 
## iter  10 value 79.672889
## iter  20 value 61.024523
## iter  30 value 52.975248
## iter  40 value 49.192357
## iter  50 value 47.888735
## iter  60 value 47.498354
## iter  70 value 47.479953
## iter  80 value 47.475037
## final  value 47.474800 
## converged
## # weights:  17
## initial  value 190.759738 
## iter  10 value 104.158952
## iter  20 value 92.275065
## iter  30 value 89.168829
## iter  40 value 87.202206
## iter  50 value 87.157743
## iter  60 value 87.118593
## iter  70 value 73.791429
## iter  80 value 71.014806
## iter  90 value 67.496849
## iter 100 value 66.384273
## final  value 66.384273 
## stopped after 100 iterations
## # weights:  49
## initial  value 183.563893 
## iter  10 value 107.285633
## iter  20 value 96.831062
## iter  30 value 92.807631
## iter  40 value 85.200333
## iter  50 value 82.248761
## iter  60 value 82.156230
## iter  70 value 82.114985
## iter  80 value 82.048337
## iter  90 value 76.602796
## iter 100 value 75.870244
## final  value 75.870244 
## stopped after 100 iterations
## # weights:  81
## initial  value 169.767186 
## iter  10 value 119.190339
## iter  20 value 107.262992
## iter  30 value 95.183260
## iter  40 value 85.914493
## iter  50 value 82.101174
## iter  60 value 79.689781
## iter  70 value 79.625704
## iter  80 value 78.634893
## iter  90 value 68.525760
## iter 100 value 63.415286
## final  value 63.415286 
## stopped after 100 iterations
## # weights:  17
## initial  value 180.628894 
## final  value 108.999997 
## converged
## # weights:  49
## initial  value 166.100775 
## iter  10 value 83.605340
## iter  20 value 75.123014
## iter  30 value 74.738197
## iter  40 value 71.888074
## iter  50 value 69.543423
## iter  60 value 69.285254
## iter  70 value 68.012842
## iter  80 value 67.668823
## iter  90 value 66.586843
## iter 100 value 66.366948
## final  value 66.366948 
## stopped after 100 iterations
## # weights:  81
## initial  value 185.358361 
## iter  10 value 101.988755
## final  value 95.000001 
## converged
## # weights:  17
## initial  value 163.585335 
## iter  10 value 83.494214
## iter  20 value 72.126381
## iter  30 value 62.299472
## iter  40 value 61.311142
## final  value 61.295697 
## converged
## # weights:  49
## initial  value 184.277871 
## iter  10 value 73.623756
## iter  20 value 59.455315
## iter  30 value 54.621712
## iter  40 value 52.718984
## iter  50 value 52.491889
## iter  60 value 52.483655
## final  value 52.483439 
## converged
## # weights:  81
## initial  value 205.815924 
## iter  10 value 69.803170
## iter  20 value 52.730147
## iter  30 value 47.744643
## iter  40 value 46.036771
## iter  50 value 44.752320
## iter  60 value 43.938336
## iter  70 value 43.778503
## iter  80 value 43.677376
## iter  90 value 43.454885
## iter 100 value 42.858678
## final  value 42.858678 
## stopped after 100 iterations
## # weights:  17
## initial  value 173.438791 
## iter  10 value 85.837670
## iter  20 value 80.198692
## iter  30 value 69.384482
## iter  40 value 68.479305
## iter  50 value 67.827094
## iter  60 value 66.489316
## iter  70 value 65.964117
## iter  80 value 65.842343
## iter  90 value 65.803430
## iter 100 value 65.130330
## final  value 65.130330 
## stopped after 100 iterations
## # weights:  49
## initial  value 150.017774 
## iter  10 value 59.950794
## iter  20 value 41.338958
## iter  30 value 34.611049
## iter  40 value 33.290139
## iter  50 value 32.559691
## iter  60 value 32.379812
## iter  70 value 32.254080
## iter  80 value 32.182197
## iter  90 value 32.144874
## iter 100 value 32.083578
## final  value 32.083578 
## stopped after 100 iterations
## # weights:  81
## initial  value 166.484864 
## iter  10 value 73.652622
## iter  20 value 62.242060
## iter  30 value 61.009795
## iter  40 value 60.121697
## iter  50 value 58.992627
## iter  60 value 57.980616
## iter  70 value 57.947639
## iter  80 value 54.200989
## iter  90 value 54.007461
## iter 100 value 53.991429
## final  value 53.991429 
## stopped after 100 iterations
## # weights:  17
## initial  value 160.724131 
## final  value 118.000000 
## converged
## # weights:  49
## initial  value 165.017142 
## final  value 120.000050 
## converged
## # weights:  81
## initial  value 233.174848 
## iter  10 value 95.704070
## iter  20 value 93.989653
## iter  30 value 93.981033
## iter  40 value 92.983507
## iter  50 value 92.968427
## iter  60 value 92.808731
## iter  70 value 86.283848
## iter  80 value 82.164939
## iter  90 value 81.903064
## iter 100 value 79.025499
## final  value 79.025499 
## stopped after 100 iterations
## # weights:  17
## initial  value 172.175885 
## iter  10 value 89.228892
## iter  20 value 75.223995
## iter  30 value 66.712820
## iter  40 value 64.317206
## final  value 64.283240 
## converged
## # weights:  49
## initial  value 211.829835 
## iter  10 value 65.483280
## iter  20 value 58.095718
## iter  30 value 54.252061
## iter  40 value 52.578703
## iter  50 value 51.751153
## iter  60 value 51.393708
## iter  70 value 51.340694
## iter  80 value 51.310647
## iter  90 value 51.247522
## iter 100 value 51.008339
## final  value 51.008339 
## stopped after 100 iterations
## # weights:  81
## initial  value 218.996302 
## iter  10 value 113.249092
## iter  20 value 68.862777
## iter  30 value 55.996824
## iter  40 value 53.136916
## iter  50 value 50.989458
## iter  60 value 46.877068
## iter  70 value 45.987400
## iter  80 value 45.745555
## iter  90 value 45.590569
## iter 100 value 45.586274
## final  value 45.586274 
## stopped after 100 iterations
## # weights:  17
## initial  value 172.713437 
## iter  10 value 98.408597
## iter  20 value 88.649300
## iter  30 value 87.358688
## iter  40 value 85.264200
## iter  50 value 85.152570
## iter  60 value 84.751217
## iter  70 value 81.750382
## iter  80 value 80.819061
## iter  90 value 80.586547
## iter 100 value 79.857368
## final  value 79.857368 
## stopped after 100 iterations
## # weights:  49
## initial  value 202.789309 
## iter  10 value 89.679421
## iter  20 value 82.543268
## iter  30 value 78.924850
## iter  40 value 76.958918
## iter  50 value 76.934378
## iter  60 value 76.106316
## iter  70 value 75.986885
## iter  80 value 75.972902
## iter  90 value 75.014041
## iter 100 value 75.003138
## final  value 75.003138 
## stopped after 100 iterations
## # weights:  81
## initial  value 201.846077 
## iter  10 value 94.422353
## iter  20 value 93.420902
## iter  30 value 93.417210
## iter  40 value 93.406088
## iter  50 value 93.000188
## iter  60 value 90.487606
## iter  70 value 80.199016
## iter  80 value 78.390987
## iter  90 value 75.636117
## iter 100 value 72.214285
## final  value 72.214285 
## stopped after 100 iterations
## # weights:  17
## initial  value 177.018334 
## iter  10 value 96.017855
## iter  20 value 95.996585
## iter  30 value 87.000263
## iter  40 value 86.999546
## iter  50 value 81.922178
## iter  60 value 81.000002
## final  value 80.999991 
## converged
## # weights:  49
## initial  value 170.994846 
## iter  10 value 78.836280
## iter  20 value 71.568728
## iter  30 value 67.230966
## iter  40 value 64.753272
## iter  50 value 62.994277
## iter  60 value 62.044455
## iter  70 value 61.341906
## iter  80 value 60.484250
## iter  90 value 60.189929
## iter 100 value 60.041808
## final  value 60.041808 
## stopped after 100 iterations
## # weights:  81
## initial  value 181.021314 
## final  value 128.000000 
## converged
## # weights:  17
## initial  value 177.479235 
## iter  10 value 102.635751
## iter  20 value 72.626377
## iter  30 value 63.891469
## iter  40 value 63.706245
## iter  50 value 63.705138
## iter  50 value 63.705137
## iter  50 value 63.705137
## final  value 63.705137 
## converged
## # weights:  49
## initial  value 200.836658 
## iter  10 value 73.112954
## iter  20 value 57.296030
## iter  30 value 53.837878
## iter  40 value 52.554542
## iter  50 value 52.271059
## iter  60 value 52.116438
## iter  70 value 52.115235
## final  value 52.115222 
## converged
## # weights:  81
## initial  value 199.236319 
## iter  10 value 92.881345
## iter  20 value 74.386269
## iter  30 value 57.710924
## iter  40 value 52.461811
## iter  50 value 47.569724
## iter  60 value 44.957588
## iter  70 value 43.780100
## iter  80 value 43.235401
## iter  90 value 42.871718
## iter 100 value 42.276413
## final  value 42.276413 
## stopped after 100 iterations
## # weights:  17
## initial  value 176.370101 
## iter  10 value 100.927829
## iter  20 value 90.566089
## iter  30 value 83.230612
## iter  40 value 80.972392
## iter  50 value 78.423569
## iter  60 value 78.396917
## iter  70 value 78.389615
## iter  80 value 78.375774
## iter  90 value 78.365475
## iter 100 value 77.618738
## final  value 77.618738 
## stopped after 100 iterations
## # weights:  49
## initial  value 177.078257 
## iter  10 value 89.384132
## iter  20 value 85.632521
## iter  30 value 85.407221
## iter  40 value 84.566559
## iter  50 value 82.491769
## iter  60 value 81.403035
## iter  70 value 80.432439
## iter  80 value 80.172188
## iter  90 value 79.572345
## iter 100 value 76.684886
## final  value 76.684886 
## stopped after 100 iterations
## # weights:  81
## initial  value 191.698154 
## iter  10 value 123.279760
## iter  20 value 123.273900
## iter  30 value 108.032194
## iter  40 value 107.177535
## iter  50 value 105.408171
## iter  60 value 105.396482
## iter  70 value 105.394517
## iter  80 value 105.392391
## iter  90 value 105.389282
## iter 100 value 102.488418
## final  value 102.488418 
## stopped after 100 iterations
## # weights:  17
## initial  value 212.195236 
## iter  10 value 109.644758
## iter  20 value 89.451273
## iter  30 value 77.892853
## iter  40 value 77.725196
## iter  50 value 76.925943
## iter  60 value 76.819738
## iter  70 value 76.802622
## iter  80 value 76.798158
## iter  90 value 76.795355
## iter 100 value 76.791410
## final  value 76.791410 
## stopped after 100 iterations
## # weights:  49
## initial  value 173.226456 
## final  value 115.999986 
## converged
## # weights:  81
## initial  value 208.552598 
## iter  10 value 71.926801
## iter  20 value 50.374039
## iter  30 value 41.921174
## iter  40 value 40.354616
## iter  50 value 38.367031
## iter  60 value 36.367199
## iter  70 value 35.561563
## iter  80 value 34.004166
## iter  90 value 33.143639
## iter 100 value 32.706404
## final  value 32.706404 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.903345 
## iter  10 value 73.831466
## iter  20 value 65.303671
## iter  30 value 64.363274
## final  value 64.363168 
## converged
## # weights:  49
## initial  value 204.664227 
## iter  10 value 101.151948
## iter  20 value 64.533699
## iter  30 value 56.546193
## iter  40 value 55.679733
## iter  50 value 55.451386
## iter  60 value 49.711290
## iter  70 value 49.279632
## iter  80 value 49.270070
## iter  90 value 49.269986
## iter  90 value 49.269986
## iter  90 value 49.269986
## final  value 49.269986 
## converged
## # weights:  81
## initial  value 217.047950 
## iter  10 value 116.562144
## iter  20 value 72.474913
## iter  30 value 55.536291
## iter  40 value 51.375358
## iter  50 value 48.737960
## iter  60 value 47.948534
## iter  70 value 47.351906
## iter  80 value 45.555243
## iter  90 value 43.868514
## iter 100 value 43.665791
## final  value 43.665791 
## stopped after 100 iterations
## # weights:  17
## initial  value 175.436876 
## iter  10 value 91.948794
## iter  20 value 65.562379
## iter  30 value 62.708342
## iter  40 value 61.279564
## iter  50 value 61.148305
## iter  60 value 61.117219
## iter  70 value 61.086230
## iter  80 value 61.077269
## iter  90 value 61.075830
## iter 100 value 61.074657
## final  value 61.074657 
## stopped after 100 iterations
## # weights:  49
## initial  value 172.037114 
## iter  10 value 106.489788
## iter  20 value 102.239443
## iter  30 value 101.413575
## iter  40 value 100.617053
## iter  50 value 96.589597
## iter  60 value 93.011893
## iter  70 value 77.239130
## iter  80 value 60.815161
## iter  90 value 60.128703
## iter 100 value 55.627716
## final  value 55.627716 
## stopped after 100 iterations
## # weights:  81
## initial  value 175.663165 
## iter  10 value 92.630547
## iter  20 value 83.035931
## iter  30 value 80.860338
## iter  40 value 79.742544
## iter  50 value 78.429552
## iter  60 value 78.125039
## iter  70 value 75.241944
## iter  80 value 61.754954
## iter  90 value 60.243539
## iter 100 value 59.663527
## final  value 59.663527 
## stopped after 100 iterations
## # weights:  17
## initial  value 171.598470 
## iter  10 value 100.978033
## iter  20 value 97.626540
## iter  30 value 93.600578
## iter  40 value 87.891871
## iter  50 value 86.398006
## iter  60 value 84.558082
## iter  70 value 83.760907
## iter  80 value 83.252915
## iter  90 value 83.023710
## iter 100 value 82.789640
## final  value 82.789640 
## stopped after 100 iterations
## # weights:  49
## initial  value 198.733874 
## iter  10 value 87.455528
## iter  20 value 69.420599
## iter  30 value 66.509718
## iter  40 value 64.006203
## iter  50 value 63.999926
## final  value 63.999811 
## converged
## # weights:  81
## initial  value 168.582420 
## iter  10 value 105.828305
## iter  20 value 87.398584
## iter  30 value 86.459700
## iter  40 value 85.800312
## iter  50 value 85.800038
## final  value 85.800036 
## converged
## # weights:  17
## initial  value 172.241772 
## iter  10 value 89.307976
## iter  20 value 65.042667
## iter  30 value 63.566036
## iter  40 value 63.479689
## iter  50 value 63.478433
## final  value 63.478426 
## converged
## # weights:  49
## initial  value 183.081581 
## iter  10 value 68.606249
## iter  20 value 59.119410
## iter  30 value 56.123174
## iter  40 value 54.771064
## iter  50 value 54.684654
## iter  60 value 54.682477
## final  value 54.682456 
## converged
## # weights:  81
## initial  value 193.099835 
## iter  10 value 72.685864
## iter  20 value 57.215628
## iter  30 value 52.725647
## iter  40 value 48.600730
## iter  50 value 47.436639
## iter  60 value 46.662376
## iter  70 value 45.811503
## iter  80 value 45.498362
## iter  90 value 45.049495
## iter 100 value 44.967879
## final  value 44.967879 
## stopped after 100 iterations
## # weights:  17
## initial  value 191.683707 
## iter  10 value 88.190388
## iter  20 value 83.650766
## iter  30 value 83.173166
## iter  40 value 83.171559
## iter  50 value 83.165821
## iter  60 value 82.198718
## iter  70 value 82.159903
## iter  80 value 79.274709
## iter  90 value 76.809318
## iter 100 value 76.140461
## final  value 76.140461 
## stopped after 100 iterations
## # weights:  49
## initial  value 196.684142 
## iter  10 value 75.293198
## iter  20 value 59.311818
## iter  30 value 49.548710
## iter  40 value 40.489549
## iter  50 value 36.519573
## iter  60 value 35.489430
## iter  70 value 35.283528
## iter  80 value 35.187042
## iter  90 value 35.149377
## iter 100 value 35.116050
## final  value 35.116050 
## stopped after 100 iterations
## # weights:  81
## initial  value 206.174585 
## iter  10 value 116.688940
## iter  20 value 115.123133
## iter  30 value 112.363388
## iter  40 value 110.857979
## iter  50 value 107.496096
## iter  60 value 105.646256
## iter  70 value 105.461480
## iter  80 value 104.452996
## iter  90 value 103.414909
## iter 100 value 103.331902
## final  value 103.331902 
## stopped after 100 iterations
## # weights:  17
## initial  value 191.645923 
## iter  10 value 82.127393
## iter  20 value 71.735780
## iter  30 value 64.954345
## iter  40 value 64.545081
## iter  50 value 63.703168
## iter  60 value 63.601899
## iter  70 value 63.540379
## iter  80 value 63.511849
## iter  90 value 63.500114
## iter 100 value 63.496840
## final  value 63.496840 
## stopped after 100 iterations
## # weights:  49
## initial  value 180.803001 
## iter  10 value 90.444022
## iter  20 value 80.830310
## iter  30 value 69.367250
## iter  40 value 66.075838
## iter  50 value 65.888656
## iter  60 value 65.020491
## iter  70 value 65.011455
## iter  80 value 65.005403
## iter  90 value 65.000848
## iter 100 value 64.996849
## final  value 64.996849 
## stopped after 100 iterations
## # weights:  81
## initial  value 173.383684 
## iter  10 value 101.751146
## iter  20 value 91.999948
## iter  30 value 91.000313
## iter  40 value 91.000083
## iter  50 value 90.999923
## iter  60 value 90.999498
## iter  70 value 90.996774
## iter  80 value 90.553389
## iter  90 value 87.001423
## iter 100 value 86.999480
## final  value 86.999480 
## stopped after 100 iterations
## # weights:  17
## initial  value 160.817726 
## iter  10 value 102.180512
## iter  20 value 66.438111
## iter  30 value 64.149006
## iter  40 value 64.039540
## iter  50 value 64.039018
## iter  50 value 64.039018
## iter  50 value 64.039018
## final  value 64.039018 
## converged
## # weights:  49
## initial  value 178.258157 
## iter  10 value 74.184370
## iter  20 value 61.391358
## iter  30 value 56.891179
## iter  40 value 54.734473
## iter  50 value 54.239443
## iter  60 value 54.162649
## iter  70 value 54.157973
## final  value 54.157804 
## converged
## # weights:  81
## initial  value 187.463367 
## iter  10 value 64.121743
## iter  20 value 55.074441
## iter  30 value 50.345913
## iter  40 value 49.535019
## iter  50 value 48.943633
## iter  60 value 48.784252
## iter  70 value 45.737253
## iter  80 value 44.803473
## iter  90 value 44.769283
## iter 100 value 44.768858
## final  value 44.768858 
## stopped after 100 iterations
## # weights:  17
## initial  value 174.399442 
## iter  10 value 108.608919
## iter  20 value 105.988435
## iter  30 value 102.640029
## iter  40 value 102.083361
## iter  50 value 98.266454
## iter  60 value 91.260006
## iter  70 value 84.532473
## iter  80 value 84.453428
## iter  90 value 79.589683
## iter 100 value 78.604146
## final  value 78.604146 
## stopped after 100 iterations
## # weights:  49
## initial  value 189.143827 
## iter  10 value 79.967263
## iter  20 value 76.074327
## iter  30 value 74.101025
## iter  40 value 70.916402
## iter  50 value 68.764222
## iter  60 value 64.522554
## iter  70 value 61.737633
## iter  80 value 56.056447
## iter  90 value 53.989374
## iter 100 value 53.269744
## final  value 53.269744 
## stopped after 100 iterations
## # weights:  81
## initial  value 157.114206 
## iter  10 value 116.788848
## iter  20 value 113.000423
## iter  30 value 101.008417
## iter  40 value 95.381601
## iter  50 value 92.763810
## iter  60 value 90.993085
## iter  70 value 90.400091
## iter  80 value 90.139641
## iter  90 value 87.398998
## iter 100 value 86.120368
## final  value 86.120368 
## stopped after 100 iterations
## # weights:  81
## initial  value 190.386439 
## iter  10 value 83.005890
## iter  20 value 61.874274
## iter  30 value 56.699084
## iter  40 value 54.851690
## iter  50 value 53.100887
## iter  60 value 52.700258
## iter  70 value 52.583348
## iter  80 value 52.348912
## iter  90 value 52.240948
## iter 100 value 52.239237
## final  value 52.239237 
## stopped after 100 iterations

plot(modell_nn6)

Auffällig bei diesem Modell mit diesen Trainingsdaten ist, dass das jeweils beste Modell mit jeweils 5 Hidden Units ist. Bei allen 3 unterschiedlichen Weights erreicht man mit 5 Hidden Units eine Trainingsaccuracy von knapp unter 92%. Das beste Modell ist mit einem Weight Decay von 0,1.

modell_nn6_best <- modell_nn6$bestTune
modell_nn6_best

##   size decay
## 9    5   0.1

predict_testNN_6 = predict(modell_nn6, test_eng_nn)
predict_testNN_6<-sapply(predict_testNN_6,round,digits=0)
nn_table6 <- table(test_eng_nn$target, predict_testNN_6)

Dieses Modell erreicht eine Accuracy von knapp unter 90% und eine Specificity von 50%. Positiv bei diesem Modell ist, dass lediglich 2 Patienten fälschlicherweise als gesund ausgegeben werden, obwohl sie erkrankt sind.

results_nn6 <- data.frame(actual = test_eng_nn$target, prediction = predict_testNN_6)
conf_nn6 <- confusionMatrix(table(results_nn6$actual,results_nn6$prediction))
conf_nn6

## Confusion Matrix and Statistics
## 
##    
##      0  1
##   0 86  8
##   1  2  9
##                                           
##                Accuracy : 0.9048          
##                  95% CI : (0.8318, 0.9534)
##     No Information Rate : 0.8381          
##     P-Value [Acc > NIR] : 0.0362          
##                                           
##                   Kappa : 0.5908          
##                                           
##  Mcnemar's Test P-Value : 0.1138          
##                                           
##             Sensitivity : 0.9773          
##             Specificity : 0.5294          
##          Pos Pred Value : 0.9149          
##          Neg Pred Value : 0.8182          
##              Prevalence : 0.8381          
##          Detection Rate : 0.8190          
##    Detection Prevalence : 0.8952          
##       Balanced Accuracy : 0.7533          
##                                           
##        'Positive' Class : 0               
## 

acc_nn6 <- conf_nn6$overall[1]
sens_nn6 <- conf_nn6$byClass[1]
spec_nn6 <- conf_nn6$byClass[2]

Explainer für das Neural Net Model:

Um die Neuronalen Netz Modelle etwas besser zu verstehen, haben wir nun noch die DALEX Library verwendet, um das NN-Modell, dass auf die Testdaten am besten abgeschnitten hat, besser erklären zu können.

library(DALEX)

## Warning: package 'DALEX' was built under R version 3.6.2

## Welcome to DALEX (version: 1.2.1).
## Find examples and detailed introduction at: https://pbiecek.github.io/ema/

## 
## Attaching package: 'DALEX'

## The following object is masked from 'package:dplyr':
## 
##     explain

#create Explainer
p_fun <- function(object, newdata){predict(object, newdata=newdata, type="prob")[,2]}

explainer_nn <- explain(modell_nn4, label = "nn", 
                                    data = data_test, y = as.numeric(data_test$target),
                                    colorize = FALSE, predict_function = p_fun) 

## Preparation of a new explainer is initiated
##   -> model label       :  nn 
##   -> data              :  105  rows  17  cols 
##   -> target variable   :  105  values 
##   -> model_info        :  package caret , ver. 6.0.85 , task Classification (  default  ) 
##   -> predict function  :  p_fun 
##   -> predicted values  :  numerical, min =  5.201416e-05 , mean =  0.1150474 , max =  0.949322  
##   -> residual function :  difference between y and yhat (  default  )
##   -> residuals         :  numerical, min =  0.05067799 , mean =  0.9897145 , max =  1.965565  
##   A new explainer has been created!

model_perf_nn <- model_performance(explainer_nn)

plot(model_perf_nn)

plot(model_perf_nn, geom = "boxplot")

## Warning: Ignoring unknown parameters: fun

## No summary function supplied, defaulting to `mean_se()

vi_classif_nn <- variable_importance(explainer_nn, loss_function = loss_root_mean_square)

Der Plot der Variable Importance zeigt, dass der Faktor sickness, den mit Abstand größten Einfluss auf die Prediction des Modells hat. Manche Blutwerte (Leukocytes, Hematocrit) hingegen haben so gut wie keinen Einfluss auf den Output.

plot(vi_classif_nn)

Nun erstellen wir noch 2 Partial Dependence Plots mit einer “unwichtigen” Variable, in unserem Fall den Leukocyten und der Variable mit dem größten Einfluss, dem Faktor ob jemand vorerkrankt ist oder nicht.

pdp_classif_nn_leuko <- variable_profile(explainer_nn, variable =  "Leukocytes", type = "partial" )
pdp_classif_nn_sick <- variable_profile(explainer_nn, variable =  "sickness", type = "partial" )

## 'variable_type' changed to 'categorical' due to lack of numerical variables.

plot(pdp_classif_nn_leuko)

Je höher der Leukocyt Gehalt im Blut, desto wahrscheinlicher ist es, dass bei unserem Modell jemand als gesund klassifiziert wird. Der Zusammenhang ist bei unserem Modell fast linear.

plot(pdp_classif_nn_sick)

Bei der Variable Sickness verhält es sich so, dass jemand ohne Vorerkrankung bei uns eher als Corona krank klassifiziert wird als jemand mit Vorerkrankung.

ale_classif_nn_leuko <- variable_profile(explainer_nn, variable =  "Leukocytes", type = "accumulated")
ale_classif_nn_sick <- variable_profile(explainer_nn, variable =  "sickness", type = "accumulated")

## 'variable_type' changed to 'categorical' due to lack of numerical variables.

plot(ale_classif_nn_leuko)

plot(ale_classif_nn_sick)

Naive Bayes Classifier

# Naive Bayes Classifier

set.seed(7267166)
trainIndex=createDataPartition(data_clean$target, p=0.7)$Resample1
train=data_clean[trainIndex, ]
test=data_clean[-trainIndex, ]

## check the balance
print(table(data_clean$target))

## 
##   0   1 
## 474  58

# Naive Bayes Classifier

library(e1071)

NBclassfier_clean=naiveBayes(target~., data=train)
print(NBclassfier_clean)

## 
## Naive Bayes Classifier for Discrete Predictors
## 
## Call:
## naiveBayes.default(x = X, y = Y, laplace = laplace)
## 
## A-priori probabilities:
## Y
##         0         1 
## 0.8900804 0.1099196 
## 
## Conditional probabilities:
##    Patient.age.quantile
## Y        [,1]     [,2]
##   0  9.259036 6.309663
##   1 14.024390 4.557893
## 
##    Patient.addmited.to.regular.ward..1.yes..0.no.
## Y            0          1
##   0 0.95180723 0.04819277
##   1 0.58536585 0.41463415
## 
##    Patient.addmited.to.semi.intensive.unit..1.yes..0.no.
## Y           0         1
##   0 0.9246988 0.0753012
##   1 0.8292683 0.1707317
## 
##    Patient.addmited.to.intensive.care.unit..1.yes..0.no.
## Y            0          1
##   0 0.95180723 0.04819277
##   1 0.90243902 0.09756098
## 
##    sickness
## Y            0          1
##   0 0.41867470 0.58132530
##   1 0.97560976 0.02439024
## 
##    Hematocrit
## Y         [,1]      [,2]
##   0 -0.1075208 0.8217811
##   1  0.2888207 0.7556972
## 
##    Platelets
## Y          [,1]      [,2]
##   0  0.03512984 0.7547930
##   1 -0.73513596 0.6355735
## 
##    Mean.platelet.volume
## Y          [,1]      [,2]
##   0 -0.02173547 0.7889366
##   1  0.13781037 0.7297791
## 
##    Lymphocytes
## Y          [,1]      [,2]
##   0 -0.05718726 0.8662317
##   1 -0.17805014 0.6997478
## 
##    Mean.corpuscular.hemoglobin.concentration..MCHC.
## Y          [,1]      [,2]
##   0 -0.05445320 0.8801287
##   1  0.09684454 0.8125258
## 
##    Leukocytes
## Y         [,1]      [,2]
##   0  0.1331424 0.8058705
##   1 -0.5721571 0.8686501
## 
##    Basophils
## Y            [,1]      [,2]
##   0 -0.0009014746 0.9948337
##   1 -0.1522362580 0.6939421
## 
##    Mean.corpuscular.hemoglobin..MCH.
## Y          [,1]      [,2]
##   0 -0.03964834 0.8006152
##   1 -0.13236144 0.9709085
## 
##    Eosinophils
## Y          [,1]      [,2]
##   0  0.05686628 0.9288753
##   1 -0.54435114 0.3297303
## 
##    Monocytes
## Y            [,1]      [,2]
##   0 -0.0002117383 0.8333964
##   1  0.3914686489 0.9893394
## 
##    Red.blood.cell.distribution.width..RDW.
## Y         [,1]      [,2]
##   0 0.12311207 0.9211221
##   1 0.05486193 1.0619366

printALL=function(model){
  trainPred=predict(model, newdata = train, type = "class")
  trainTable=table(train$target, trainPred)
  testPred=predict(model, newdata=test, type="class")
  testTable=table(test$target, testPred)
  trainAcc=(trainTable[1,1]+trainTable[2,2])/sum(trainTable)
  testAcc=(testTable[1,1]+testTable[2,2])/sum(testTable)
  message("Contingency Table for Training Data")
  print(trainTable)
  message("Contingency Table for Test Data")
  print(testTable)
  message("Accuracy")
  print(round(cbind(trainAccuracy=trainAcc, testAccuracy=testAcc),3))
}
printALL(NBclassfier_clean)

## Contingency Table for Training Data

##    trainPred
##       0   1
##   0 320  12
##   1  12  29

## Contingency Table for Test Data

##    testPred
##       0   1
##   0 137   5
##   1   8   9

## Accuracy

##      trainAccuracy testAccuracy
## [1,]         0.936        0.918

Naive Bayes auf den engineerten Daten

train <- read.csv("data/clean/train_feat_eng.csv")
test <- read.csv("data/clean/test_feat_eng.csv")

NBclassfier_eng <- naiveBayes(target~., data=train)
#printALL(NBclassfier_eng)

conf_nb <- confusionMatrix(table(test$target, predict(NBclassfier_clean, test)))

Tree based Models

library(rpart)
library(rpart.plot)
library(randomForest)

## randomForest 4.6-14

## Type rfNews() to see new features/changes/bug fixes.

## 
## Attaching package: 'randomForest'

## The following object is masked from 'package:ggplot2':
## 
##     margin

## The following object is masked from 'package:dplyr':
## 
##     combine

data_clean <- read.csv("data/clean/data_clean.csv")
set.seed(3456)
train_idx <- createDataPartition(data_clean$target, p = .8, 
                                  list = FALSE, 
                                  times = 1)

data_train <- data_clean[train_idx, ]
data_test <- data_clean[-train_idx, ]

train_feat_eng <- read.csv("data/clean/train_feat_eng.csv")
test_feat_eng <- read.csv("data/clean/test_feat_eng.csv")

#In einem ersten Versuch verwenden wir das Paket rpart und die originalen Featrues unseres Datensatzes.
set.seed(200989)
trees1_fit <- rpart(target ~., data = data_train, method = "class")

#Plot des ersten Fits
rpart.plot(trees1_fit)

Hier sehen wir, wie auch in anderen Modellen, dass die Leukozyten besonders wichtig für die Beurteilung / den Ausschluss einer Infektion sind.

#Suche nach dem minimalen Fehler
min_cp <- trees1_fit$cptable[which.min(trees1_fit$cptable[,"xerror"]),"CP"]
min_cp

## [1] 0.01

#Pruning des Trees
trees1_prune <- prune(trees1_fit, cp = min_cp)
trees1_pruned_test_prediction <- predict(trees1_prune, newdata = data_test, type = "class")

cf1 <- confusionMatrix(trees1_pruned_test_prediction, as.factor(data_test$target))
cf1

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction   0   1
##          0 101   1
##          1   2   2
##                                           
##                Accuracy : 0.9717          
##                  95% CI : (0.9195, 0.9941)
##     No Information Rate : 0.9717          
##     P-Value [Acc > NIR] : 0.6472          
##                                           
##                   Kappa : 0.5571          
##                                           
##  Mcnemar's Test P-Value : 1.0000          
##                                           
##             Sensitivity : 0.9806          
##             Specificity : 0.6667          
##          Pos Pred Value : 0.9902          
##          Neg Pred Value : 0.5000          
##              Prevalence : 0.9717          
##          Detection Rate : 0.9528          
##    Detection Prevalence : 0.9623          
##       Balanced Accuracy : 0.8236          
##                                           
##        'Positive' Class : 0               
## 

tree1_acc <- cf1[["overall"]][["Accuracy"]]
tree1_spec <- cf1[["byClass"]][["Specificity"]]
tree1_sens <- cf1[["byClass"]][["Sensitivity"]]
tree1_prec <- cf1[["byClass"]][["Precision"]]

Als Ergebnis der Anwendung unseres ersten Baummodels bekommen wir mit 97% Accuracy, 98% Sensitivity und einer Spezifität von rund 67% bereits sehr gute Ergebnisse. Im nächsten Schritt versuchen wir, das Ergebnis mit der Verwendung der von uns erstellten Features zu verbessern.

#Neuer Versuch mit neuen den neuen Features
trees2_fit <- rpart(target ~., data = train_feat_eng, method = "class")
trees2_prediction <- predict(trees2_fit, newdata = test_feat_eng, type = "class")
rpart.plot(trees2_fit)

#summary(trees2_fit)

#Test Confusion Matrix für den tree mit den angepassten Features
cf2 <- confusionMatrix(trees2_prediction, as.factor(test_feat_eng$target))
cf2

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction  0  1
##          0 80  3
##          1 14  8
##                                           
##                Accuracy : 0.8381          
##                  95% CI : (0.7535, 0.9028)
##     No Information Rate : 0.8952          
##     P-Value [Acc > NIR] : 0.97525         
##                                           
##                   Kappa : 0.4012          
##                                           
##  Mcnemar's Test P-Value : 0.01529         
##                                           
##             Sensitivity : 0.8511          
##             Specificity : 0.7273          
##          Pos Pred Value : 0.9639          
##          Neg Pred Value : 0.3636          
##              Prevalence : 0.8952          
##          Detection Rate : 0.7619          
##    Detection Prevalence : 0.7905          
##       Balanced Accuracy : 0.7892          
##                                           
##        'Positive' Class : 0               
## 

tree2_acc <- cf2[["overall"]][["Accuracy"]]
tree2_spec <- cf2[["byClass"]][["Specificity"]]
tree2_sens <- cf2[["byClass"]][["Sensitivity"]]
tree2_prec <- cf2[["byClass"]][["Precision"]]

Wir sehen nun, dass sich die Specificity auf Kosten der Accuracy und Sensitivität verbessert - insgesamt jedoch schlechtere Ergebnisse liefert.

Sehen wir nun, ob wir unsere ersten Ergebnisse mit einem Random Forest Modell verbessern können.

#Bagging mit einem RF mit den originalen Features
set.seed(200989)
rf1_fit <- randomForest(as.factor(target) ~ ., data = data_train, mtry = 2, importance = TRUE, ntrees = 220, type = "classification")
rf1_prediction <- predict(rf1_fit, newdata = data_test, type = "class")
cf_rf1 <- confusionMatrix(rf1_prediction, as.factor(data_test$target))
cf_rf1

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction   0   1
##          0 103   1
##          1   0   2
##                                           
##                Accuracy : 0.9906          
##                  95% CI : (0.9486, 0.9998)
##     No Information Rate : 0.9717          
##     P-Value [Acc > NIR] : 0.1949          
##                                           
##                   Kappa : 0.7954          
##                                           
##  Mcnemar's Test P-Value : 1.0000          
##                                           
##             Sensitivity : 1.0000          
##             Specificity : 0.6667          
##          Pos Pred Value : 0.9904          
##          Neg Pred Value : 1.0000          
##              Prevalence : 0.9717          
##          Detection Rate : 0.9717          
##    Detection Prevalence : 0.9811          
##       Balanced Accuracy : 0.8333          
##                                           
##        'Positive' Class : 0               
## 

rf1_acc <- cf_rf1[["overall"]][["Accuracy"]]
rf1_spec <- cf_rf1[["byClass"]][["Specificity"]]
rf1_sens <- cf_rf1[["byClass"]][["Sensitivity"]]
rf1_prec <- cf_rf1[["byClass"]][["Precision"]]

Das Random Forest Modell liefert uns mit einer Accuracy von 99% und einer und einer Spezifität von 67% extrem gute Ergebnisse. Sehen wir nun, ob uns die neuen Features noch etwas bringen:

#Random Forest mit den neuen Features
set.seed(200989)

rf2_fit <- randomForest(as.factor(target) ~ ., data = train_feat_eng, mtry = 15, 
                        importance = TRUE, ntrees = 100, type = "class")

rf2_prediction <- predict(rf2_fit, newdata = test_feat_eng, type = "class")

cf_rf2 <- confusionMatrix(rf2_prediction, as.factor(test_feat_eng$target))
cf_rf2

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction  0  1
##          0 92  3
##          1  2  8
##                                           
##                Accuracy : 0.9524          
##                  95% CI : (0.8924, 0.9844)
##     No Information Rate : 0.8952          
##     P-Value [Acc > NIR] : 0.03059         
##                                           
##                   Kappa : 0.7355          
##                                           
##  Mcnemar's Test P-Value : 1.00000         
##                                           
##             Sensitivity : 0.9787          
##             Specificity : 0.7273          
##          Pos Pred Value : 0.9684          
##          Neg Pred Value : 0.8000          
##              Prevalence : 0.8952          
##          Detection Rate : 0.8762          
##    Detection Prevalence : 0.9048          
##       Balanced Accuracy : 0.8530          
##                                           
##        'Positive' Class : 0               
## 

rf2_acc <- cf_rf2[["overall"]][["Accuracy"]]
rf2_spec <- cf_rf2[["byClass"]][["Specificity"]]
rf2_sens <- cf_rf2[["byClass"]][["Sensitivity"]]
rf2_prec <- cf_rf2[["byClass"]][["Precision"]]

Wie auch schon beim Tree Modell verschlechtern sich hier die Ergebnisse mit den neuen Features.

Zu guter letzt versuchen wir noch den besten Tune mit Hilfe einer zufälligen Suche zu finden. Das Suchsetup nutzt 15-fache CV und 3 Wiederholungen

# Zufällige Suche nach dem richtigen Setup
control_trees <- trainControl(method="repeatedcv", number=15, repeats=3, search="random")
set.seed(200989)
mtry_trees <- sqrt(16)
rf_random <- train(as.factor(target)~., data=data_train, method="rf", metric="Accuracy", tuneLength=15, trControl=control_trees)

print(rf_random)

## Random Forest 
## 
## 426 samples
##  16 predictor
##   2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (15 fold, repeated 3 times) 
## Summary of sample sizes: 397, 399, 397, 399, 398, 398, ... 
## Resampling results across tuning parameters:
## 
##   mtry  Accuracy   Kappa    
##    1    0.8868434  0.1832382
##    3    0.9257658  0.5690219
##    5    0.9305044  0.6111560
##    6    0.9305317  0.6127604
##    8    0.9320917  0.6240885
##    9    0.9335969  0.6269844
##   10    0.9343632  0.6337862
##   11    0.9328032  0.6285357
##   12    0.9328032  0.6269938
##   13    0.9311571  0.6205962
##   15    0.9295131  0.6185752
## 
## Accuracy was used to select the optimal model using the largest value.
## The final value used for the model was mtry = 10.

plot(rf_random)

Wie schlägt sich dieses Modell mit mtry 10 nun bei der Vorhersage der Test-Daten?

rf_random_prediction <- predict(rf_random, newdata = data_test)
cf_rf_rand <- confusionMatrix(as.factor(rf_random_prediction), as.factor(data_test$target))
cf_rf_rand

## Confusion Matrix and Statistics
## 
##           Reference
## Prediction   0   1
##          0 103   1
##          1   0   2
##                                           
##                Accuracy : 0.9906          
##                  95% CI : (0.9486, 0.9998)
##     No Information Rate : 0.9717          
##     P-Value [Acc > NIR] : 0.1949          
##                                           
##                   Kappa : 0.7954          
##                                           
##  Mcnemar's Test P-Value : 1.0000          
##                                           
##             Sensitivity : 1.0000          
##             Specificity : 0.6667          
##          Pos Pred Value : 0.9904          
##          Neg Pred Value : 1.0000          
##              Prevalence : 0.9717          
##          Detection Rate : 0.9717          
##    Detection Prevalence : 0.9811          
##       Balanced Accuracy : 0.8333          
##                                           
##        'Positive' Class : 0               
## 

rf_rand_acc <- cf_rf_rand[["overall"]][["Accuracy"]]
rf_rand_spec <- cf_rf_rand[["byClass"]][["Specificity"]]
rf_rand_sens <- cf_rf_rand[["byClass"]][["Sensitivity"]]
rf_rand_prec <- cf_rf_rand[["byClass"]][["Precision"]]

Auch dieses Setup liefert und die gleichen Ergebnisse wie der ursprüngliche Random Forest mit den originalen Features.

Zum Abschluss sehen wir uns noch einmal die Übersicht der Ergebnisse an:

library(kableExtra)
modell_trees <- c("Tree 1", "Tree 2", "RF 1", "RF 2", "RF rand")
tree_test_acc <- c(tree1_acc, tree2_acc, rf1_acc, rf2_acc, rf_rand_acc)
tree_sens <- c(tree1_sens, tree2_sens, rf1_sens, rf2_sens, rf_rand_sens)
tree_spec <- c(tree1_spec, tree2_spec, rf1_spec, rf2_spec, rf_rand_spec)
results_trees = data.frame(
  "model" = modell_trees,
  "sensitivity" = tree_sens,
  "Specificity" = tree_spec,
  "Accuracy" = tree_test_acc
)

kable_styling(kable(results_trees, reesformat = "html", digits = 4), full_width = FALSE)

model	sensitivity	Specificity	Accuracy
Tree 1	0.9806	0.6667	0.9717
Tree 2	0.8511	0.7273	0.8381
RF 1	1.0000	0.6667	0.9906
RF 2	0.9787	0.7273	0.9524
RF rand	1.0000	0.6667	0.9906

Wir haben gesehen, dass insbesondere die RF Modelle auf unsere Test Daten sehr gute Ergebnisse liefern. Wie jedoch bereits beschrieben, liegt hier ein “rare cases” Problem vor und es bleibt abzuwarten, wie die Modelle in anderes balancierten Datensätzen performen.

Beste Modelle:

`

modell <- c("SVM Radial Sigma Kernel", "Neural Network", "Naive Bayes Classifier", "Random Forest")
accuracies <- c(CM_RS$overall[1], acc_nn4, conf_nb$overall[1], rf1_acc)
sensitivities <- c(CM_RS$byClass[1],sens_nn4, conf_nb$byClass[1], rf1_sens)
specificities <- c(CM_RS$byClass[2],spec_nn4, conf_nb$byClass[2], rf1_spec)
results_overall = data.frame(
  "model" = modell,
  "sensitivity" = sensitivities,
  "Specificity" = specificities,
  "Test Accuracy" = accuracies
)

kable_styling(kable(results_overall, format = "html", digits = 4), full_width = FALSE)

model	sensitivity	Specificity	Test.Accuracy
SVM Radial Sigma Kernel	0.9789	0.9000	0.9714
Neural Network	0.9474	0.6000	0.9143
Naive Bayes Classifier	0.8842	0.0000	0.8000
Random Forest	1.0000	0.6667	0.9906

In der Übersichtstabelle kann man sehr gut erkennen, dass sowohl das beste SVM Modell als auch das beste Tree Modell eine sehr hohe Accuracy von über 97% erreichen. Auch das Neuronale Netz erreicht nich eine gute Accuracy von über 91%. Der Unterschied zwischen den Modellen wird aber sehr stark bemerkbar wenn man auf die Specificity schaut, diese ist beim SVM mit Radial Kernel und getuntem Sigma Parameter deutlich besser als bei allen anderen Modellen mit 90%. Dieses Modell würde sich eignen für die Klassifikation von Corona Infizierten und Nicht Coronainfizierten. Das neuronale Netz und der Random Forest mit etwas mehr als 60 % Specificity schneiden hierbei schlechter ab und würden wir deshalb weil es sich hier um sehr sensible Gesundheitsdaten handelt nicht “produktiv” verwenden. Der Naive Bayes Classifier schneidet bei uns nur leicht besser ab als das Base Modell aus dem Proposal, die logistische Regression.

Schlussfolgerung:

Wie nach der Explorativen Datenanalyse erwartet, ist bei den verschiedenen Modellen schwierig sowohl eine hohe Specificity als auch eine hohe Sensitivity zu erreichen. Unsere Modelle erzielen durchwegs eine hohe Sensitivität, wobei nur ein Modell eine Spezifizität von über 70% erreicht hat. Die Accuracy ist meistens auf einem hohen Niveau und die Unterschiede sind ziemlich gering zwischen den einzelnen Modellen. Die hohe Sensitivity ermöglicht ein relativ präzises Ausschließen von Nicht-Coronainfizierten, welches beim Testen ein wichtiger Faktor ist, um tatsächlich Infizierte besser behandeln und identifizieren zu können. Unser weiteres Vorgehen wäre, einen weiteren Test zu implementieren, der auf etwas anderen Faktoren beruht, um im Anschluss an unsere jetzige Klassifikation, die False Positives von den True Positives besser unterscheiden zu können. Im Großen und Ganzen kann man sagen, dass wir ein Modell gefunden haben, dass deutlich besser als 50:50 abschneiden würden beim Klassifizieren von Corona Patienten und dies ist bereits sehr viel Wert. Trotzdem muss gesagt werden, dass wir bei einer solch sensiblen Klassifikation auf jeden Fall ein nachgelagerten Test noch empfehlen würden.